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ABSTRACT 


An investigation is made of the Navy Exploratory Develop- 
ment Program; its newly revamped management system, its 
organization and underlying rationale. The investigation 
then addresses the processes of program selection, review and 
evaluation as applied to technology base programs, followed 
by an overview of a methodology called “Evaluation Research" 
as it applies to goal oriented programs. The paper is dir- 
ected toward determining the effectiveness of technology base 
programs, with a major emphasis on the changing structure of 
the Exploratory Development Program within the Department of 


Defense. 
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To ODUeCL TON 


There has never been a period in history in which the 
determination of the worth of programs within the Department 
of Defense (DoD) structural framework has been so critically 
viewed. Adding to this observation is the discernible fact 
that relative budgets within the Research and Development 
(R&D) Program are steadily decreasing. 

This concern encompasses total system acquisition, oper- 
ations and maintenance, hardware and software development, 
concept feasibility, pure research and all other programs 
intermediate. Further, the concern is broad-based to the 
extent that it has become a prime distress not only to the 
government sector of DoD, but also to the supporting indus- 
trial counterparts. 

Owing to the desire then to better justify on-going pro- 
fees, tO promote innovation and concentration on highest 
leverage opportunities, to expedite the transfer of technolgy 
and projects to advanced development (category 6.3) and use 
of new technical capabilities developed in research (category 
6.1) and to demonstrate relevance and cost effectiveness, 
the Research and Development program in total is currently 
undergoing a modification of the existing management planning 
System, Particularly the Exploratory Development Program (EDP). 

The objective of this paper then is really twofold: 
meeto investigate the changing structure, organization and 


underlying rationale of the Exploratory Development Program 





in the Navy and (2) to investigate the processes of program 
selection, review and evaluation as applied to technology 
programs within the EDP. Included will be a brief on the 
development of the EDP, a treatise on why it is a necessary 
and contributing element of R&D and also a discussion of what 
the future might hold for determining the "worth" of technol- 
ogy type programs. 

Because of the significance placed on an understanding 
of the operations of the EDP as a necessary and crucial pre- 
reguisite to developing a meaningful evaluation methodology, 
the majority of this paper is focused on identifying the 
foundations and workings of the newly revamped Exploratory 
Development Program. This understanding is essential before 
any succinct attempt can be made to alleviate the discrepan- 
cies in the review and evaluation techniques, existing today. 
Hopefully, a closer interpretation of the EDP will facilitate 
a more viable methodology for determining the Return-on-Invest- 


ment (ROI) of future technology programs within DoD. 
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Ii. BACKGROUND 


Pee lus LECHNOLOGY BASE 

Somewhere between basic Naval research on one end of the 
Research and Development (R&D) spectrum and full scale devel- 
opment on the other, fall a great many projects which come 
under the heading of applied technology, or technology base 
programs. The technology base is the total reservoir of 
organized knowledge from directly and indirectly sponsored 
basic research into physical and social phenomena and the 
feasibility of new processes, techniques and components for 
uSing them (Ref. 1). Its end result is new organized knowl- 
edge. The base also retains fallout information and contri- 
butions left over from exploration and development of specific 
systems. The creation and exploration of candidate systems, 
in turn, is shaped by the information available from the tech- 
nology base. 

One of the largest and perhaps most dynamic businesses in 
the United States today is that dealing with the acquisition 
of major weapon systems for the Department of Defense. Due to 
the size and complexity of the federal government and its 
associated agencies, management of the affairs of this organi- 
zation requires a tremendous amount of activity and coordination. 

This business entity, not unlike any other, maintains a 
growth posture made possible by both internal and external 
influences. These influences are visible through the technol- 


ogy development programs supported by the organization. Thus, 
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a most important faction of the R&D community within the 
Department of Defense (DoD) is that responsible for developing 
and maintaining a strong technology base. 

The technology base is supported by both Government and 
private activities. Private groups (such as industrial com- 
panies) and nonprofit organizations (such as Federal contract 
research centers and universities) both contribute to the 
technology base and recommend technology base programs, just 
as do the federal laboratories and other supporting agencies. 

Within DoD, the defense laboratories are charged with main- 
taining the technology base by sponsoring activities in support 
of potential systems. Laboratory research objectives are 
planned to generate better technological tools five to twenty 
years in the future. There does indeed exist a requirement 
for the technology to be related to mission/operational needs. 

Due to the seemingly nebulous nature of these technology 
oriented programs, a great deal of scrutiny has been, and con- 
tinues to be, given them, with primary emphasis of the concern 
aimed at their resultant value. Because of the presence of 
intangible outputs in the sense of quantification, coupled 
with an evidence of decreasing relative budgets, technology 
administrators at all levels within the command have been, and 
continue to be, faced with the dilemma of determining which 
programs should receive their attention. Their problem is 
essentially twofold: (1) How are they to determine which of 
the many proposed (alternative) programs should be selected 
for support, and (2) how are they to evaluate the effectiveness 


of their programs? 
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Beeic Very ed) -ilemicecomxmow how much shovild be spent for 


technology base activities because payoffs are virtually 
unpredictable and only rarely are they realized at some future 
date in a directly traceable manner (Ref. 1). Operational 
needs can be met most effectively at the lowest possible cost 
by innovative products built on new technology being created 
for its own sake (pure technology) as well as that which 
remains as fallout from other system efforts (applied 
eechnology). 

A significant problem in defense has been to control the 
relationship between technology base activities and system 
acquisition programs. Technology base projects have gone on 
to develop more than just the raw elements and material to 
meet as yet unexpressed needs. Traditionally they have devel- 
oped subsystems and system concepts independently of specific 
needs. Once developed, subsystems have sought out homes in 
new system candidates and systems in turn have searched for a 
need they could satisfy. Further, these subsystems have been 
old technologies extended to improve the performance of old 
Ranas of products. 

There 1S a common misconception that defense systems have 
suffered because they have embodied too much "new technology" 
and, as a result, have become too complex, sophisticated and 
expensive (Ref. 1). The explanation for this is that what is 
"new" about "new technology" is the higher level of perform- 


ance demanded from the old technical approaches. In fact, 
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subjective ratings have indicated that systems, while becoming 
more complicated and expensive, have become less technologi- 
cally advanced. 

Applying new technology to meet more stringent needs can, 
in fact, result in simpler, less costly systems. As a common 
example, the transistor development has markedly reduced the 
cost and size of electronic systems and improved their reli- 
ability, maintainability and other cost of ownership factors. 

The need for an active and viable technology base is thus 
extremely important. The larger the span of technical and 
scientific activity sponsored by the Government, the more 
extensive will be the optional technological choices avail- 
able to the system designer. It iS imperative that cost- 
benefits be maximized in providing the technology base for 
new systems. This then can only be realized when the base 
program is driven from operational needs and mission require- 
ments. This, in turn, requires a most judicial and expedient 
methodology for determining the criteria for the selection 
and evaluation and the effectiveness of the technology base 
programs in the R&D environment. 

Just as critical as a well defined and operable methodology 
for determining selection and evaluation criteria, is the 
understanding of the actual program, or system, which is to 
be operated on. No attempt could ever be made to determine a 
program's worth, or return on investment (ROI), without a prior 
and thorough understanding of the functions and underlying 


philosophies of that program. It is crucial that a sound 
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foundation be established from which goals and strategies 
can be derived rather than the traditional dollar value of 


Sutcpuc. 


be LHE EXPLORATORY DEVELOPMENT PROGRAM AS THE CRITICAL LINK 

While many definitions of exploratory development exist, 
there are several functions that should be fulfilled by this 
category of Research and Development. These might be grouped 
into the following areas; 

1. The translation and application of research results 
POmoOcaICwned ll wce, Including the formulation of new 
technology areas. 

2. Feasibility demonstrations of component and system 
hardware concepts. 

3. Applications of new technology to existing components 
or systems. 

The nature of the preceding functions impiles that both the 
inputs and outputs of exploratory development are quite 
varied, and can be thought of as a continuum. Maximum benefit 
in an Exploratory Development organization would appear to 
feecue [rom having the continuum functions in close proximitv. 

The purpose of the Exploratory Development program as 

delineated by NAVMATINST 3910.7B of 3 October 1974 is as 
follows: 
The purpose of the Exploratory Development program 

is to develop the technological wherewithall to solve 

Specific Navy and Marine Corps problems, and to identify 

and exploit new technological opportunities which may 

stimulate new operational concepts within the operating 


forces. It is the policy of the Chief of Naval Develop- 
ment that the program include concept formulation effort 
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in the form of analytical and experimental effort to 
help identify problems, determine alternative solu- 
tions thereto in terms of prospective systems, sub- 
subsystems and techniques, and demonstrate the technical 
feasibility of those solutions to a degree which 
warrants their consideration for support under Advanced 
Development. Selectively, where a system or subsystem 
is relatively small and of low technical risk, the 
program may support effort aimed at acquiring informa- 
tion prerequisite to commencement of Engineering 
Development. Such effort, however, should stop short 

of engineering refinements characteristic of proto- 
typing development for procurement, inventory or opera- 
tional deployment. While the program may also encompass 
analytical and experimental work on the technologies 
directly related to materials, components, processes, 
techniques, and individual equipments of Navy and 

Marine Corps interest, broad-based investigations in 
physical, biological, environmental, medical, behavioral, 
social and engineering sciences characteristics of the 
Naval Research program are not permitted. Effort of 

an exploratory nature, which has a primary purpose of 
direct support of specific Advanced, Engineering or 
Operational Systems Development projects should be 
funded and accomplished as an integral part of those 
projects, subject to close coordination with related 
effort in the Exploratory Development program. 


The operation of this system has several characteristics 
that are significant in the management of applied technology 
meategory 6.2) programs (Ref. 2). 


® Very broad technological base requiring judgements and 
technical expertise in many fields at all levels of control. 


e Many layers of control--DDR&E, ASN (R&D), CND, SYSCOMs, 
Offices, DLP, DNT, laboratories--and necessity for coordina- 
tion of tasks with related programs as well as with pure 
research (category 6.1) and advanced development (category 
peo) . 


® Coordination necessary between services and with the 
industrial complex. 


® Results of program efforts are seldom visible or recog- 
nized in the system that reaches operational use. 


® Categorization of tasks is not easily relatable to naval 
responsibilities, missions, operational concepts, or systems. 
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@e CNO's guidance on Naval needs is not directly trans- 
latable into R&D requirements. CNM does not interpret oper- 
ational needs for SYSCOMs. Each SYSCOM (and suborganization 
within SYSCOM) employs a different subjective means in 
defining R&D goals. There is no priority assigned to indi- 
vidual tasks. 


e The system creates a large number of low level funded 
tasks (approximately 650 Task Area Plans (TAPs) (DD1634) 
yearly). 

@ Approved TAPS can be underfunded or not funded at the 
discretion of SYSCOMs or laboratories without reference to 
emp or DNT. 

@e Reductions in funding are frequently made by propor- 
tionate cuts which results in perpetuation of sub-critical 
programs and reduced effort on critical programs. 


e Excessive effort is devoted to making minor changes in 
funding levels with resulting turbulence. 


e Inordinate effort 1s devoted to management, funding 
changes and justifications of programs which takes technical 
personnel away from productive tasks. 


e The system fosters "Micro-management." Program reviews 
frequently do not result in "make-a-difference" changes. 


@ Protection of in-house effort emphasized as contrasted 
memcOntcractor effort. 


@e Delay in exploiting available technology. 
@e Overemphasis on evolutionary and fix-it projects. 


@e Inadequate scientific and technical advice in special- 
ized fields involving programs of several organizations. 


e Critical problems exist related to coupling the 6.1, 
6.2, and 6.3 programs to initiate an Engineering Development 
mo.4) program. 

fe thus becomes evident that a crucial prerequisite to 
any understanding of the methodologies of Exploratory Develop- 
ment program selection and evaluation is a thorough under- 
Standing of the program itself. For this reason, and owing 


to the fact that EDP is at this time undergoing major organi- 


Ziclomnal and structural revision, a Significant amount of 
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attention is shown this subject in this thesis. The EDP is 
indeed the “critical link" in terms of developing major weapon 
systems within the Department of Defense, and understanding 
the functions and operations of it are also critical in terms 
of measuring program worth. The changes in the program have 
been initiated in an effort to alleviate the negative aspects 
of the previously mentioned characteristics of the program 
Seperation. It is felt that only through an understanding of 
the function/operational aspects of the EDP can the concept 
of Return-on-Investment be addressed, since this must ulti- 
mately be linked to the organizations' goals and strategies 


rather than dollar value of output. 
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IfL. EXPLORATORY DEVELOPMENT PROGRAM 


A. A RATIONALE FOR A POLICY PLATFORM 

Because of the tremendous size and complexity of the 
Federal Government and its many associated agencies, manage- 
ment of the affairs of this organization constitutes a very 
Significant amount of activity and coordination. Accordingly, 
a great deal of scrutiny is given the policy platforms and 
expenditure characteristics within sub-organizations of this 
complex. 

This section will investigate the Exploratory Development 
Program within the Department of the Navy, examine the policy 
of the EDP and the rationale behind that policy, examine 
several influences which are evidenced by the technology pro- 
gram, and provide a personal critique of that policy plat- 
merm Currently existing for the EDP from the standpoint of a 
euelic procurement. 

It is felt that a comfortable understanding of the EDP 
policy is a valuable and key element which allows a better 
comprehension of the significance of the EDP structural 
change, thus leading toward a discussion of the present pro- 
gram organization and characteristics. The approach then 
Will be to first address the broader issue of external policy 
influences and then focus on particular program policies as 
delineated by the Assistant Secretary of the Navy for Research 


and Development (ASN (R&D)). 
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l. The EDP as a Public Program 


If any government program could be labeled "public," 
the Exploratory Development Program must certainly be con- 
sidered as a leading contender. The business of this program 
is certainly conducted for the general welfare of the people 
of the United States through the guise of building and main- 
taining a strong and viable defense posture. Clearly then, 
the output of this entity is to be labeled a public good, 
Since it is consumed by all members of the community of the 
United States. The control of expenditures is governed by 
the internal processes within the Department of Defense (DOD), 
Since the costs of production cannot be maintained by competi- 
mive Pricing. 

Because of the nature of the product in the Explora- 
tory Development Program, the citizens do not actively par- 
face ln setting policy or in providing an oversight function. 
This task is left to entrusted officials within DoD, as well 
as the mix of activities performed. The financing aspect of 
the EDP comes under close scrutiny through the auspices of 
both Congress and the Office of Management and Budget (OMB) 
in the Executive Branch of government. Programs on-going 
must continually be justified and reviewed for content, return, 
applicability and need. 

There is a facet of the EDP which presents itself as 
an externality to the private sector. This is the area which 


is embodied by the notion of "technology transfer." This 


term 1s used to imply that technology, as the output from a 
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research effort, can be utilized productively in many loca- 

tions and organizations over and beyond its initial specific 
mission (Ref. 3). The process of technology transfer in the 
EDP has traditionally been one of primarily outflux from DoD. 

The working policy and structural framework of the 
EDP is built to closely resemble a standard "rational model" 
Organization (Ref. 4). This will become quite evident later 
when the working policy/critique is presented. However, 
implementation of this policy 1s probably more closely repre- 
sentative of the classical "bureaucratic model," with only 
a moderate trend toward a politically contrived system (Ref. 
5) 

It should be submitted that no "Operational Code" 
exists which might be any more distinctive than the academic 
models just mentioned. This hypothesis is supported by 
noticing the well defined organizational concepts and general 
propositions under which the EDP functions; high degree of 
standardized procedures, uncertainty avoidance, centralized 
policy formulation, decisions made by leaders, presence of 
parochial priorities, actions are organizationalized, change 
is definitely incremental, long range planning is prevalent, 
goals and tradeoffs are of primary concern, decisions are 
based on administrative feasibility, change is directed from 
higher authority and there is limited flexibility in the line 
functions of the organization. Additionally, one can very 
obviously witness well defined rules of the game, actions 


which come about as a result of political inducements, definite 
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separation and identification of chiefs and indians and much 
mis-communication, all of which point toward a strong influ- 
eiece OL the political model characteristics (Ref. 4). 

Having provided an overview of the structure of the 
EDP and its operational characteristics and public attributes, 
attention now will focus more closely on the particular policy 
Sethe program On which its platform is built. Any discus- 
Sion of the Congressional and Presidential influences on 
the EDP has been negated because it is not felt to be a pro- 
gram, primarily because of its size and technical complexity, 
which lends itself to the normal control imposed by those 
forces. All interactions which do exist are enacted at the 
ASN (R&D) level, the results of which follow. 

2. EDP Rationale 

A review of the Navy's Exploratory Development Pro- 
gram by the Assistant Secretary of the Navy (R&D) led to the 
recent concern to re-establish the policies which are intended 
to govern that section of Research and Development. As a 
result, the ASN (R&D) has specified the policies, performance 
objectives, and responsibilities concerned with the overall 
management of the Department of the Navy Exploratory Develop- 
ment Program (Ref. 6). 

It is the belief of the ASN (R&D) that improvements 
are possible in technical strategy, responsiveness to innova- 
tion, transferring results into useful applications, visi- 
bility to management, and demonstration of the rationale and 


cost effectiveness of the Exploratory Development Program and 
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its elements. The underlying rationale of the Exploratory 
Development Program is derived from the fact that this 
activity, like all others, must compete for its resources. 
This implies that the programs be continually justified in 
meaningful and convincing terms. 

According to the ASN (R&D), to assure that the long 
term benefits are clear, Exploratory Development must demon- 
strate how it (Ref. 6): 

a. Provides technical opportunities which stimulate 

the development of substantially improved and/ 


or totally new operational capabilities. 


b. Provides solutions to problems arising in the 
developmental process. 


c. Provides opportunities to regain operational 
Superiority or to exploit and enhance an 
existing operational superiority. 

d. Provides for the transformation of new research 
discoveries into useful applications, and for 
the innovative extension of existing technologies 
to Significantly more advanced applications. 

Se Ve OlLlicyerormulation 
Using as a foundation the principles implicit in the 
current operation of the Program, policies have been identi- 
Fied which are intended: to enhance the productivity of 
Exploratory Development; and to improve the ability to demon- 
Siemaece the need for, and benefits resulting from, the pro- 
gram's activities (Ref. 6). 

a. The Exploratory Development Program shall be 

based on a set of dynamic technical strategies 


designed to: 


(1) take maximum advantage of new technical 
Opportunities 


(2) exploit deficiencies in opposition capa- 
bilities, and 
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(3) provide prompt response to perceived 
requirements for superior naval capability. 


b. The Exploratory Development Program shall be 
made visible through an information system per- 
mitting technical, as well as financial audit 
on a current basis to cognizant levels of 
management. The principal intent of the inform- 
ation system is to permit control of the tech- 
nical strategy at higher levels, facilitate 
execution of the programs at the most techni- 
cally competent levels, and facilitate the 
transfer of technology to advanced development. 


c. The Exploratory Development Program shall include, 
PORES Max<iImuMneextent practicable, "block 
programs’ which are identified with major tech- 
nical strategies and under the management of an 
organization whose mission/product line or 
technical competence is most appropriate. 


d. The Exploratory Development Program plan shall 
be consistent with overall assigned ratios of 
in-house to contract work for balanced utiliza- 
tion of the national technological base. 


e. The Exploratory Development Program and its 
elements shall be based on a more explicit 
application of the return-on-investment (ROT) 
CONnCGeDiE. 


f. The Exploratory Development Program shall be 
based on cost-effective principles and shall 
consider, as well as budgetary implications, 
the potential costs of acquisition and ownership. 


g. The Exploratory Development Program shall be 
based on providing adequate funding for effective 
exploitation of highest-leverage opportunities. 
When over-all resources are inadequate or 
reduced, funds required for support of such 
projects shall be made available through can- 

Co ap One clbtaliment OL projects with lower 
promise of military worth within the framework 
of technical strategy. 


4. An Assessment 
In general, the rationale and the policies identified 
are not inconsistent with the overall policies of Fiscal and 


Life Cycles of Defense Systems. That is to say, the basic 
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questions of need, technical feasibility and funding avail- 
ability are inherent in the policy information for the 
Exploratory Development Program. However, an immediate con- 
cern which comes to mind is whether the policies, being 
quite broad and abstract by their very nature, can be sup- 
ported and/or even interpreted at the working levels. This 
@Sencerm iS not peculiar to this particular policy formula- 
tion, rather it deals with the structure of policy enactment 
through any large organization. 

Because the policies have already been identified, 
the approach that will be taken now will be to address each 
policy statement separately and provide personal comment as 
deemed appropriate. 

Policy A: Primarily the emphasis should be placed 
on the term technical strategy, as a means of assuring that 
the programs supported are productive, needed and will pro- 
vide benefit. It would appear that a sophisticated and 
well-managed communication (information flow) scheme 1s 
necessary to carry out the desires of top management. Worth 
questioning is the productivity, however, of focuSing atten- 
meon on the deficiencies of the opposition to the detriment 
Sewencouraging an offensive approach. Further, to provide 
for ‘prompt response’ would necessarily imply that the cor- 
porate memory and developmental capability of the in-house 
sector be maintained. 

Policy B: Almost without exception would there be 
an agreement with the notion of a needed buildup of a more 


meaningful information system. It is always potentially 
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possible though that a problem of unidirectional information 
flow will ultimately come into being. There also appears to 
be some conflict with the idea of maintaining control at the 
higher levels, and at the same time encouraging decentraliza- 
tion of the EDP. Decentralization is essential in order to 
promote and produce technical excellence and thus realize 
maximum productivity. 

PesacyG:. There currently exists much support of the 
notion of ‘block programming’ on the basis of less duplica- 
tion of efforts, greater knowledge of state-of-the-art in the 
particular product lines, and increased ability to select and 
evaluate the individual programs within the EDP. 

Policy D: Clearly the bottom line here is to bolster 
and maintain a strong technology base, whether that be pro- 
vided for by industry or in-house. An obvious question which 
surfaces given a predetermined ratio, is just where in the 
Organization that ratio must be measured--for this will have 
a definite influence on accountability within the system. It 
should be realized that the ultimate decision should be one 
which yields the maximum advantage to the government's carry- 
ing out its mission. This philosophy necessarily requires a 
consensus that those goals and missions have properly and 
adequately been delineated. 

Policy E: While the notion of return-on-investment 
appears relatively straightforward at first glance, its appli- 
cation in an area such as Exploratory Development is extremely 


MeeErcule £O enact. This is due to the obvious lack of 
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tangible, and thus quantifiable, output from the individual 
programs. However, in the context of wanting to maximize 

the benefits of the dollars invested, the concept is crucially 
appropriate. 

Policy F: As much as is possible, attention should 
be given to the potential downstream costs of acquisition 
and ownership. It is this basis which will ultimately provide 
for a rational decision as to which programs will be supported 
in future Exploratory Development programs. 

Policy G: Certainly the philosophy most often 
followed in practice would lead one to select those alterna- 
tives yielding the greatest leverage. Seriously questionable, 
however, is that portion of this policy that suggests each 
program be evaluated on its own merits and would be subject 
to cancellation if it showed less favor in terms of military 
work. It is of general belief (supporting that belief exis- 
ting at the NAVMAT level) that in most cases, existing pro- 
grams will continue to be funded unless there is obvious 
reason to alter that decision on the basis of either lessened 
Or increased need. 

5. Concluding Comments 

There is a general trend in the Navy's Research and 
Development Program, including the Exploratory Development 
Program, to shift back to an originally supported organiza- 
tional structure of being decentralized. This is due toa 
large extent to the desire to promote accountability at all 


levels of the R&D organization. Additionally, it is observed 
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that trends definitely indicate future total budgets to rise, 
while the percentage allocation to the Exploratory Develop- 
ment category will, as has been experienced in the past 
decade, definitely decline. As a result of this observation, 
mato noted that technology development is definitely not 
independent of weapon system development. 

It is noteworthy in summary to exploit the under- 
lying principles evidenced in the policy formulation for the 
Exploratory Development Program: 


@e Supportive of technological advances of funda- 
mental importance to operational needs. 


® Quest for technical and managerial excellence at 
all levels. 


@® Recognition of the necessity for both top-down 
Gucecevon and DOCEOm=up Inputs. 


® Accountability essential at all executive levels. 


e Necessity for strategies as tools to implement 
policies. 


@® Recognition that sometimes technical advance must 
be injected into the mainstream of the system 
ens CuLodl ae 


@® Recognition of essential need for adequate 
information systems to allow interaction. 


While it is a rather straightforward task to outline 
Beeeaeronal approach, it is extremely difficult, if not 
impossible, to encounter a purely rational organization in 
operation. Influences, personalities, and constraints are 
entirely too numerous to allow anything other than some form 
of an “individualized" organization to exist today, whether 
that organization be public, private or some mix of the two 


extremes. 
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An investigation of the EDP policy does, however, 
serves to provide the foundation on which its organization 
must be structured. Further, this foundation (rationale 
end policy) provides the impetus behind the current EDP mod- 
imerecation, which is the subject matter of the following 
section of this thesis. It is important that the reader be 
aware that in addition to gaining a better understanding of 
the "real" system in operation today, the material presented 
will hopefully provide a context in which to review the sub- 
jects of program selection and evaluation in applied tech- 


nology in a more meaningful manner. 


Bee Slt RUCTURE OF EDP (A CHRONICLE) 

The creation of the offices of the Director, Defense 
Research and Engineering (DDR&E), Assistant Secretary of the 
Navy (Research and Development), and Deputy Chief of Naval 
Operations (Development) (DCNO(D)) back in 1959 set the stage 
for some revolutionary changes in the way R&D programs, and 
hence Exploratory Development programs, would be planned, 
justified and implemented for the next two decades. The 
concept of a complete R&D spectrum with basic research on the 
One end and full scale development on the other was inherent 
in the procedures of the first decade. 

Technology programs were established which covered a wide 
variety of efforts, ranging from pure research in components 
and subsystems to fairly sophisticated experimental hardware. 


These technology programs at that time were characterized as 
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being comprised of projects more closely focused on end- 
products than research but entailing a higher level of uncer- 
tainty and lower financial commitment than full scale 
development. 

Bureaus and offices were established in areas relevant to 
their assigned responsibilities. Project listings were 
required to be structured according to the format of Table I 
Meer, /). So for the first time, senior staffs in the Depart- 
ment of the Navy and the Office of the Secretary of Defense 
had a viable framework for scrutinizing the exploratory devel- 


opment effort and the funds allocated to it. 


Table I Exploratory Development 


Functional Areas, 1962 


Target Surveillance 
Navigation 

Environmental Surveillance 
Integrated Surveillance 
Command Control 
Communications 

Data Processing 

Weapons and Ordnance 

Guided Missiles 

Jamming and Deception 

Naval Defense Applications 
Peer teach ne nerart SUDPOrt 
Ships and Submarines 

Boats and Amphibious Vehicles 
Logistics 

Personnel Administration 
ies tin Line 





The process of planning and justifying programs in the 
early 1960s is broadly depicted in Figure 1 (Ref. 7). There 


was a trend in the early years for the bureaus and offices to 


30 








S,096T Ataeq 
‘Se S00 GBUOLICOLITISOLS pu SUL uuen a leas Os da 


“Te Ocha 


ft 420/ROS ae tee 





ol 





employ a “carte blanche" attitude to initiate new programs 

in response to general requirements. This philosophy began 
to erode, however, with the enactment of PPBS, and eventually 
the organization became extremely controlled, and hence, cen- 
tralized by the end of the first decade. 

In 1970, DDR&E then introduced two new types of documents 
which more clearly stressed the concern of program worth, 
justification and relationship to mission areas. They were 
called Technology Coordinating Papers and Area Coordinating 
Papers. They were intended to highlight program gaps, over- 
laps, and need for improved coordination among the Services 
feet. 8) . 

And so, during the early baseline period, technology pro- 
grams represented a segment of the Navy's R&D program in 
which the management operated with little or no outside inter- 
ference. Guided only by the most general requirements, bureau 
personnel relied heavily on Fleet and laboratory personnel, 
discussions with the Fleet and experience of their own offi- 
cers and engineers to identify problems, initiate programs 
and evaluate their on-going projects. 

There was a trend toward greater emphasis and dependence 
on the Navy laboratories, i.e., block programming. The bureau/ 
SYSCOM point of view initially was to oppose block programming 
on the grounds that it amounted to delegating authority to 
the laboratories without commensurate accountability for ful- 
Filling SYSCOM responsibilities. The idea of block programming 
has since become a reality, and is now in full effect, with 


mMearly full support, in the Navy organization. 
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Block programming, as interpreted today, is a means for 
providing technical management in the laboratories and Systems 
Commands with the necessary authority (including control of 
funds) to plan and manage technical effort related to the 
pursuit of significant objectives. As will be seen later, 
block programming is a Significant influence in the current 
planning and justification process proposed for the Explora- 


tory Development Program. 


C. A CHANGING CONTEMPORARY PROGRAM 

The Navy Exploratory Development Program is just nearly 
completing a transition from a system, which has been in 
operation for a number of years and is relatively well known 
by the Navy R&D community, to a new system. Central to these 
modifications is the use of dynamic "technical strategies," 
designed to focus exploratory development more directly and 
visibly on Department of the Navy operational needs. The 
first implementation of the program utilizing these technical 
strategies will commence during the current fiscal year. 

This program reorganization is a result of the CND/DCNM(D) 
having been directed by the ASN (R&D) to establish and imple- 
ment a program to improve the exploratory development system, 
to improve higher echelons' understanding of the system and 
to justify 6.2 expenditures. 

As indicated previously, under the former 6.2 management 
system the program consisted of a large number of low level 
funded tasks which were negotiated primarily between staff 


personnel of the SYSCOMs and the laboratories without 
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direction of either top level management or the best qualified 
technical personnel who are generally located at the labora- 
tories. The relationship between funded tasks and operational 
needs was not altogether visible and the contributions that 
new technologies could make to increase effectiveness or 
decrease costs were not apparent. As a result, justification 
of the program to Congress, OSD and to ASN (R&D) was not con- 
vinceing. There was a lack of funding focus on high payoff 
innovation and on feasibility demonstration of critical end 
item products. Further, there had been an inadequate inter- 
Meetbon Of 6.1 with 6.2 and coupling with 6.3 and a lack of 
coordination with related work by other services and industry. 
The objectives of the modified 6.2 management system were 
then identified to (Ref. 9): 
@ Provide top-down direction by the use of technical 
strategies, prioritized technical objectives and 
definition of major program thrusts designed to meet 


operational requirements. 


e Provide visibility by relating the program to Navy 
functions and to operational needs. 


® Promote innovation and concentration on highest 
leverage opportunities. 


@® Expedite transfer of projects to advanced development 
(6.3) and use of new technical capabilities developed 
in research (6.1). 
e Demonstrate relevance and cost effectiveness. 
These objectives were to be achieved by clear definition 
of organizational responsibilities, development and execution 


of programs to achieve approved technical objectives, allo- 


Sarton Of blocks of funds to organizations and delegation of 
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authority to implement programs to meet assigned responsibili- 
ties. CND is the designated responsible entity to ASN (R&D) 
for operation of the system. Appendix A is included to delin- 
eate the EDP management responsibilities, including design 
principles and roles of the various organizations in the 6.2 
Pimogram . 

There are four basic elements of the new 6.2 planning sys- 
tem which utilize the technical strategy concept. 

I. Direction by Technical Strategy 

Designated strategists identify, define and analyze 
mission needs and on-going or potential technical efforts 
related to satisfying those needs. Mission needs are derived 
from analyses of threat, preferred operational concepts and 
system and subsystem concepts. Current programs and work 
by industry and other services are major factors in assessing 
technical potential. Technical objectives are derived from 
mission needs and potential technology. The strategies then 
are used to define the desired major thrust in prioritizing 
the technical objectives. 

These dynamic technical strategies are designed to 
take maximum advantage of new technological capabilities, 
exploit deficiencies in opposition capabilities and provide 
prompt response to perceived requirements for superior naval 
Capabilities (Ref 6). Basically, strategic planning is not 
a new idea, and it implies "... the process of determining 
the major objectives of an organization, and the policies and 
strategies that will govern the acquisition, use and disposi- 


tion of resources to achieve those objectives" (Ref. 10). 
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The enactment of an Exploratory Development program 
policy based on the idea of technology strategies is a signif- 
icant breakthrough in terms of developing a more meaningful 
rationale for program/project evaluation. The primary reason 
for this is that heretofore, organizations had neglected to 
recognize that there had been a definite lack of clarity in 
just what they were trying to measure. This belief is sup- 
ported by various studies of productivity measurement and pro- 
gram evaluations (Ref. 11). 

It should be recognized that Exploratory Development, 
like every other activity, must compete for its resources. 
Although the necessity for an extensive program is obvious to 
those within the RDT&E establishment, to assure maintenance 
at the desired level, the program must continually be justi- 
fied in meaningful and convincing terms. To assure that long 
term benefits are clear, Exploratory Development thus must 
demonstrate how it: 

a. provides opportunities which stimulate the 

development of improved or totally new 
operational capabilities, 


b. provides solutions to problems arising in 
the development process, 


Cc. provides opportunities to regain operational 
Superiority or enhance existing superiority, 


d. provides for the transformation of new research 
discoveries into useful applications, and the 
extension of existing technologies into signifi- 
cantly more advanced applications. 

To accomplish this, then, the system being assembled 

at the NAVMAT 03 level addresses not only objectives, strate- 


gies and tactics (OST), but also interrelates them and 
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describes how the system works to harness research and devel- 
opment for the achievement of definite organizational goals 
Wxet. 12). 

Essentially the OST system is based on three major 
ideas: (1) formulation of an OST plan consisting of a hier- 
archal arrangement of corporate objectives, technical strate- 
gies, and developmental tactics, (2) the super position of 
this organization over the normal operating organization, 
and (3) the use of line managers in the operating organiza- 
tion as action officers. Figure 2 presents this system ina 
somewhat simplified manner. 

This current model thus contains all the elements of 
the historical and modern approaches, e.g., cost, timeliness, 
risk, weapon systems, value measures, etc., and also provides 
the additional and quite significant notion that the goals 
and policies of the organization need be accounted for. 

The strategies then, as developed for the Navy EDP, 
will indicate allocations recommended for additional objectives 
with justification. DNT will provide guidance regarding over- 
Bet concepts for functional areas, and also to strategists 
through the inclusion of relevant material from program objec- 
tive memorandums. 

2. Negotiating the Program 

With the implementation of the new policy, all explor- 
atory development programs will now be prenegotiated between 
SYSCOMs, laboratories and DNT prior to beginning each fiscal 
year. After the programs have been developed they then are 
recommended to DNT for allocation of funds. 
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There are three types (categories) of programs, the 
Characteristics of which determine how the organization will 
manage them. Cat I is broadbased, multi-use technology. It 
is negotiated between DNT and laboratory CO/TDs and funded 
and managed by the laboratories. All "Special topic" strategy 
areas are included in Cat I programs. Cat II is concept- 
related effort for which the SYSCOMs are customers. It is 
concerned primarily with exploratory development of systems 
and subsystems concepts for accomplishing mission needs. 
Projects for this kind of work are negotiated between the 
using SYSCOMs and the cognizant laboratories. Once negotiated 
and approved by DNT, it is funded directly as a block to the 
performing laboratory which manages it. Cat III work is per- 
Eermed Primarily by industry or other contractors. It in- 
volves primarily work related to transition of concepts to 
advanced development, and it is managed directly by the SYS- 
COMs. Figure 3 shows the "chain of accountability" which now 
exists in the EDP and the types (categories) or programs for 
which the various tiers are responsible. The planning pro- 
cess which has been formulated in consert with the new manage- 
ment system is depicted in Figure 4. 

3. Allocation of Funds 

Of the total funds allocated for the EDP, the Chief 
of Naval Development withholds a total of 15 percent of each 
program element to take care of subsequent Congressional cuts, 
to provide for new starts, for developing the strategies, for 


IR/IED, MAT 03 management, the Energy Office and DNT's special 
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FIGURE 4. 
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megiecteS. tne bulk of the Eunds (roughly $2.75B yearly) 
are allocated to SYSCOMs and laboratories. Once the con- 
tracts for work have been negotiated, changes can be made 
Pinoy ehe SYSCOMs for the CAT III programs and TD of labor- 
atories for CAT I and II programs. Final approval, as before, 
always rests with the Director of Navy Technology. 

4. Review and Evaluation 

The review and evaluation focuses on two major 

issues: (a) were the resources actually utilized as covered 
in the approved program as amended and (b) to what degree 
were the technical objectives developed by the strategies 
accomplished. It was perceived by management that the basic 
review and evaluation questions should focus on contributions 
to future Navy operations and cost effectivensss. At the 
time of research gathering for this paper there were no de- 
tails available as to how the review would be conducted but 
the objective is clear--substantive evaluation from the view- 
point of Congress, DDR&E and ASN (R&D) and not micromanagement. 
It 1S anticipated by EDP management that program administra- 
tors, strategists, and a qualified panel of independent 
individuals with knowledge of Navy needs, and other Federal 
and industrial efforts would be involved in review and evalu- 
ation process. However, it should be noted again that no 
clear and definitized methodology yet exists to conduct a 
complete program selection, review and evaluation program 


(Ref. 9). 
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5. Operation--Correlation of Elements and Strategies 

In development of the strategies, the program element 
titles, which had been originally listed by the CND, were 
expanded to the current total of 37 (Ref. 9). Although this 
increase facilitated submission by the cognizant internal 
agency, its effect is to fracture the program and make more 
difficult their aggregation in an overall plan. Figure 5 
shows the relationship which now exists between program ele- 
ments (PEs), technical strategy areas and the Science and 
Technology Objectives (STOs). It is noted that there is a 
high degree of correlation between PEs and strategies except 
in sea control and power projection where the strike warfare, 
countermeasures and two other program elements are split be- 
tween sea control and projection. 

The Chief of Naval Development establishes and promul- 
gates the structural framework for planning, programming and 
budgeting the EDP (Appendix B). This structure provides the 
link with DoD Program elements, defines the scope of work 
within specific areas, and identifies the organization within 
the Navy responsible for the planning, programming and 
budgeting. Appendix C is included to provide clarification of 
the fundation of the Exploratory Development program by delin- 
eating its new structural framework. 

The system described above is new this current fiscal 
fear. Lt is quite naturally going to involve a great deal of 
"negotiating" between the staffs of NAVMAT, laboratories and 


SYSCOMs, along with DNT, CND, and ASN (R&D). It is essential 
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that a firm understanding of the technical strategies, func- 
tional areas and program elements be had to augment an under- 
standing of the EDP objectives and transition steps. Only 
through a thorough knowledge of the inner-workings of the new 
system will any meaningful review, evaluation and assessment 
methodology be realized. As indicated, no established pro- 
Gram or technique currently exists, but there certainly is 
potential for developing a viable methodology, particularly 
with the well structured and goal-oriented EDP which exists 


Eoaay. 
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IV. PROGRAM SELECTION AND EVALUATION IN NAVY 


EXPLORATORY DEVELOPMENT 


With a background presented of the Navy Exploratory 
Development program, its functions, its responsibilities and 
its policies for implementation, it is beneficial to investi- 
gate now the manner in which programs have traditionally been 
reviewed, selected and evaluated. It would serve the reader 
well to keep in mind the new EDP characteristics, particularly 
while going through this review. This will provide a deeper 
comprehension of why the methodologies to be discussed were 
considered viable, as well as why they eventually were evi- 
denced as not being able to adequately predict, as a manage- 


ment tool, as originally intended. 


A. BACKGROUND 

Technology base programs constitute a large number of 
projects in the systems acquisition process, falling some- 
where between pure research on one end of the R&D spectrum 
and full scale development on the other. In the early years 
of the Navy R&D sector, these technology programs operated 
With little or no outside interference to the performing 
Organization. Ideas for solving Fleet related problems and 
exploiting technological opportunities were converted into 
exploratory development projects with relative ease (Ref. 7). 

In these early stages of R&D project plans, task assign- 


ments and monitoring procedures were generally flexible and 
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tailored to the needs of the individual managers at all 
levels. This process was in keeping with the "cradle to 
grave" concept. 

More recently, a perceived need to provide guidance for 
technology programs was reflected in documents ranging from 
top level Joint Chiefs of Staff strategic planning documents 
to OPNAV's General Operational Requirements and CND Explora- 
tory Development Goals. But this guidance soon tended to 
be either too general, providing little guidance at all, or 
so specific that it inhibited exploration of relevant 
alternatives. 

Because the problems associated with this guidance and 
direction quite naturally are felt at all levels within the 
organizational structure, the problem of effective program 
selection and evaluation has thus remained a significant 
issue, particularly in the area of exploratory development. 

According to a recent House Armed Services Committee 
report, concern was expressed that "... just spending money 
On research and development does not guarantee a sound R&D 
program or good military hardware" (Ref. 13). The committee 
further indicated that the Department of Defense could 
improve its efforts and its management through better tailored 
programs which would: 

e Select out weapons not cost effective 
@ Avoid unnecessary duplication of effort 
@e Place greater emphasis on product improvement 


@ Provide more meaningful management review processes. 
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Tt is not surprising that a tremendous amount of attention 
is currently being given to the valuation of DoD programs 
in Research and Development in general and Exploratory Devel- 
Soment in particular. 

Adding now to the complexity of this situation is the fact 
that throughout the past fifteen years, while the total budget 
for RDT&E,N appropriations has been steadily increasing, the 
relative percentages allocated to the technology base pro- 
grams (6.2 category) have been steadily decreasing. This 
Situation is depicted in the bar chart representation of 
apportionments, shown as Table II. 

And so, just within the past decade, many scientific, and 
some not so scientific, approaches have been tried for poten- 
tial application to technology base program evaluation and 
determination of return-on-investment figures. Most of these 
scientific techniques have been tried more at the lower 
echelons of the Navy R&D organization, althaugh few have seen 
any real application or effectiveness. The approach currently 
in vogue at the NAVMAT 03 level is built around the concept 
of "technology strategies." There appears to be merit in 


pursuing this technique, at least for a selected application. 


Eee oLloslORICAL APPROACHES 

One of the early comprehensive attempts to sort, categorize 
and evaluate R&D selection and program evaluation techniques 
was that done by Baker and Pound back in the mid-sixties (Ref. 
14). This article points out that most were oriented toward 


the operations research-management scientist, but it does 
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Table II 


Research and Development Categories as 
Percent of Total RDT&E,N Appropriation FY62-FY73 
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ae 

6.4 

B 1% 

‘hae 2 nee aaaseae 
‘en B 6.2 peecmes 
Eo = a 
petasent = qe=5 
; 6.154 4 6.1 § : i 
A 9% F 4 S% 9 44% U 


1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 


FISCAL YEAR PROGRAM 


Note: These amounts are derived from the past year columns of the various Congressional 
Budget Submissions and are not precisely consistent with New Obligational Authorizations. 
The difference results from reprogramming between program years, permitted in “‘no-year” 
appropriations. The data for each fiscal year’s program is the latest category distribution 
developed for budget presentation and best reflects actual relative dollar size of the various 
categories for each fiscal year. FY 1962 was the first year in which this category classifica- 
tion scheme was systematically applied. 


Source: Thomas Leckey, Assistant Comptroller for Budget and Reports ONR, in letter of 
24 December 1974. 
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provide a sound base for presenting the early methods thought 
to have potential. It is interesting to note that a review 
of the majority of the documents reviewed in this account 
covering this early era indicated that very few of the tech- 
niques presented actually ever saw any relevant usage. 

Keeping in consonance with the early organization and 
Management of the exploratory development sector of R&D, pro- 
ject selection thus remained a relatively unstructured prob- 
lem. For the most part, the early methods just did not 
provide an adequate description of the R&D process. At this 
point, it might appear feasible to question not only the pre- 
dictive models but the process which the models were attemp- 
ting to simulate. While it was recognized that program 
selection was a sequential decision making process, practi- 
cality would certainly indicate that detailed criticism of 
these early models would be unwarranted. 

One important observation made, according to Baker, was 
that “...there has not been a satisfactory definition of the 
objectives against which a research program should be evalu- 
ated." The significance of this statement, while having 
Clarity at this point, will take on new interpretations later 
in this section. Another key issue raised in these early 
studies, although it received little more attention than having 
been mentioned, was the idea that the "Strategic need" some- 
how ought to fit into the organizational and market structure. 

The methods applied in the early stages of investigation 


consisted almost entirely of three major methods: decision 
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theory, economic analysis and operations research. Table III 
is included in an attempt to Summarize the significant features 


of these methodologies. 


Taore i: L 
Features of Common R&D Planning Models 


Feature Method 


Decision Econanic Operations 
@nacackeriStilce consideration 





theory analysis research 

1. Sequential decisions No No Yes 
meee Profit maximization Partially Yes Yes 
PeeeeUncertainty (probability 

of success) Yes Yes Yes 
4. Relationships between criteria No Partially Yes 
5. Relationships between projects No No Partially 
6 Optimal allocation to project No Partially Yes 
7. Budget constraint Yes Yes Yes 
Beeeokill & facility constraints No No No 
wee 6 otrategic need" for project Partially No No 
10. Competitor efforts No No No 

Area of applicability 

1. Basic research Yes Maybe No 
*2. Applied research Maybe Yes Maybe 
3. Development Gt tte le Moderate Definitely 


*Technology base programs. 


A prime observation of these methods and practices is that 
there exists a strong and definite need for testing to demon- 
Strate feasibility of the predictive models. Without adequate 
demonstration, even the "best" descriptive models prove sus- 


pect under the close scrutiny of management. 
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C. MODERN APPROACHES 

Once the idea that the management process of project 
selection and evaluation was realized to have potential as 
being predictable from a management scientist approach, a 
profuse amount of literature emerged, quite naturally. A 
great many new ideas were postulated, some unique in their 
own right, although most were merely extensions of what had 
already been presented in the early years. 

An excellent study conducted by IDA provides an informa- 
tive synopsis of the modern techniques being employed (Ref. 
15). Although prepared in early 1972, the methods discussed 
are still being utilized, if only under slight improvement. 

Because the modern techniques have tended to align them- 
selves in the more strict, quantitative approach to program 
selection and evaluation, most of the discussion at this 
point will tend to follow along with that momentum. With the 
ensuing budget constraints, as previously mentioned, the 
patter associated with the modern approaches will necessarily 
allude to resource allocation as its primary objective. 
Realize, however, that the real issue is still the process of 
determining which programs are to be selected for support and 
how, then, can they be evaluated once initiated. 

Some problems are inherently embedded in the modern 
methodologies, Owing to their strong alliance to the quanti- 
tative approach of evaluation. A strong underlying problem 
is that even with modern conveniences and sophisticated ana- 


lytical software and hardware, the utilization factor still 
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Penis low Roe G meeAccording EO Albala in his work on 
Stage approaches to R&D projects: 

This condition can be partly ascribed to the fact 
that (a) most of the proposed methods involve models 
requiring much quantitative input, not readily avail- 
able within the company, and (b) in many instances 
the methods are based on techniques much too elaborate 
for the R&D manager's routine use. 

In subtle opposition to this belief is the posture taken 
fyeeltozzo Of IDA (Ref. 15.) Accordingly, Trozzo feels that: 

Dee topiewlmon Meme quantrtative methods for allo- 
cating resources within Exploratory Development should 
continue, however, pecause their rigorous structure 
should provide (1) a useful frame of reference for 
organizing information and regular liaison among system 
Specialists, technologists, and research managers, and 
(2) a systematic procedure for searching out a develop- 
ment program that will best fulfill Defense goals. 
Albeit, included as Table IV a summary deScription of the 

methods used for allocating resources, taken directly from 

the study by IDA. Its inclusion is useful in showing the 
definite trend to quantitative rationale and also to demon-~ 
strate that each organization utilizes a methodology unique to 
its particular operation, based to a large extent on its own 
needs and variables. 

Thus the techniques that have grown to occupy currently 
acceptable standards have adhered to the general conclusions 
O£ the earlier studies. To wit, there is a general recognition 
that each particular stage of R&D requires a set of inputs and 
constraints unique to its decision making process (Ref. 16). 
Another important observation of modern approaches is their 


recognition that models need be commensurate with the quality 


and quantity of data available. Finally, project success, to 
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Table 1V 
Summary Description of Methods for 
Allocating Development Resources. 








Primary Objective Control Variables 


Value Measures Operational 
Requirements 
t 


Usefulness or con- Major jobs of the 
tribution of program | Services 
results 















Method Devise an exploratory 
development program 


thet: 


Management's levers 

























Resembles the pattern 
of technological de- 

velopment in U.S, in- 
dustry 


Industrial 
Analog 


Funds spent in major 


Industry groupings-- 
technical areas 


weapon categories 
































2. TORQUE Develops technologies 
having maximum "mili- 


tary value” 


Funds to be spent in 
Next year on project 
tasks 


Subjective scaling of 
relative "military 
value" 


Operational capa- 
bility objectives 
(OCO's) assigned 
values 






















Naval Ordnance 
Laboratory 







Develops technologies 
having maximum 

"value" for future 
Navy needs 


Funds to be spent on 
each technology in 
next year 


Subjective scaling of 
relative value for 
each factor 


Warfare categories-- 
targets--military 
functions assigned 
values 































Bee. Flight 
Dynamics 
Laboratory 


Develops technologies 
having maximum "mili- 
tary value" for the 

Air Force 


Funds and manpower to 
be devoted to each 
exploratory develop- 
ment task 


Subjective scaling of 
relative effective- 
ness, timing. ,ecc. 


Operations within 
different types of 
wars--assigned values 
































Tasks trat will be 
performed 


Consists of tasks 
with maximum "ex- 
pected value" to the 
Army 


Cornell 
Aeronautical 
Laboratory 


"Essentiality" of 
weapons developed 


No general opera- 
tional requirements. 
Background to qual- 
Ltative materiel 
development objec- 







tive (QMDO) 
6. Hercules Contributes most to Funds to be spent on | Profits generated by |Market characteris- 
Corporation future company profits! individual develop- development work tics 


ment tasks 


= a es es 


7. Army Missile 


lege preferred Weapons and funding Discrete priorities | Operational require- 
Plan weapon concepts pro- of requisite tasks among weapons and iments in documents by 

jected for future technical work CDC 

deployment 
















Provides the technol- 
ogy required for prob 
able future systems 


A.F. Directorate 
of Laboratories 
Plan 


Technical effort in 
each area 






None Desired capabilities 
defined by OSD and 
the Air Staff 


Army Research 
Plan 






Responds to long Funcing emphasis for | Priorities Future operational 
range concepts and elements of program Needs in 56 opera- 
objectives of the Cional capability ob- 
Army of the future jectives 


OO 


10. Another Service | Supplies technical Funding of individual | Subjective scaling of | 29 operational re- | 
Method advances for ful- development projects merit of requirements (quirements--merit 
filling overall and contribution of weights 

Service mission technical objectives 














54 





Rapley (Cont d) 





Structural Features and Factors Considered 





Factors and Their Relationships 


Weapon Systems Development Tasks Costs 





































Method Instruments for per- | Composition of Work to achieve de- Rescurces consumed in 

forming jobs systems sired composition work 

Industrial Individual weapons-- | Weapon-product life- None DoD expenditures in 

Analog industry products times and technical Gl .2 and cota. 
(DoD expenditures-- complexity RDT&E on technical . 
industry sales) field--company funded | 

R&D in industry 
TORQUE Alternative weapons Technological ra- Technical objectives Yearly and total 














Sequenced in each 
technical require- 
ment 


quirements cf each 
weapon--criticality 
to system rated 


funding fore each 


for each OCO 
project task 


























Naval Ordnanca Contained in explor- | Technology pacing implicit--different Aggregate expendi- 






































































Laboratory atory development parameter linked to levels of pacing tures on technology 
goals (EDG) military | EDG by relevance parameters related to index of 
worth measure ; advance for parameter 

AR.F. Flight Future A.F. flight Technologies linked Confidence level Funding for differens ! 

Dynamics vehicles from tech- to systems by ap- gauge of achievement; | levels of effort | 

Laboratory nological war plan-- | plicability factor; contribution value | 
weighted for each Separate technology links task to systems 
operational re- goals and technological 
quirement goals 

Cornell QMDO description of Combinations of Ordered tasks makeup | Direct funds and 

Aeronautical hardware. Weighting | teclrmologies de- technical approach other resources 

Laboratory proposed scribe possible resource pattern needed 
technical approaches 

to subsystems 
Hercules Projected systems Substitute and com- Technical goals of Funding for each pos- 
Corporation using technology; plementary tech- specific organized sible level of effort 





contract research-- units of effort 


sales, margins 


nolegies in each 
system 






Systems Coma fae 
operational require-~ 
ments--priorities 

t 


Army Missile 


Technical composition 
Plan 


of system and subsys- 
tem concepts; general 
functional capabiii- 

ties 


Manpower, funding for 
planned tas«s 


Labs plan tasks for 
technical goals; 

tasks rated by con- 
ET SueLon 











\ 











System COnceDe pos-~ 
sibilities--most 
probable systems-- 


A.F. Directorate 
of Laboratories 
Plan 


Technical planning 
opjectives--technical 
gaps for systems; 


Summary of resources 
for technical ef- 
forts--program ele- 


Program elements-- 
Brojects-—-casxs-— 
technical efforts 



















subsystems technical goals ments 
Army Research Not exo le1t--con- Potential relevance 237 progcets in 6.2 Funding trends and 
Plan tained in long range | of technologies in alements; extra Army | planned support of 
technological fore- 30 6.2 elements to erfort elements--adequacy of 
each OCO support 
= . . ° ° e 
Another Service | None Technical objectives | 750 tasks--weights Estimated funding re- 
Method related to opera- reflect contribution | quired to complete 
tional requirements of each to each oper- } project 


ational requirement 
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Table lV (Cont'd) 


Timing 


Method 


ms 


e-phasing of re- 





Jd wi 
sources end results 

me Industrial Non= 

Analog 

2. TORQUE Weapon ICC--tech- 
Nical objectives-- 
runding (cimeliness 
UNC c2on ) 

3. Naval Ordnance Not explicit--prog- 
Laboratory ress possible in 

ae budg2t year 
! 

4. A.F. Plight Timing of technical 
Dynamics objective, weighted 
Laboratory by timeliness func- 

E20n 

5. Cornell — Timing of QMDO's 
Aeronautica). tasks 
Laboratory 

6. Hercules Timeliness of market- 
Corporation ing and time required 










for technical goals 


Army Missile 
Plan 







Pimgag of tunds., 
tasks, target dates 
Of systems 


A.F. Directorate 
of Laboratories 
Plan 








Desired capabilities 
set ‘out in five year 
increments 


Army Research 
Plan 





Operational require- 
ments out to 20 
years; development 
procram in next 1-5 
years 








Another Service 
Method 





Projects set up to 
Support operational 
requirements of 1982 












Risk 


Constraints 


Limits on possible 
program options 













Probability of actual 
outcomes differing 
from expected 


: 
None Total budget 
None 


O° 
Totai funds, in- 
house and centract 
engineers 






Probability that task 
output will fit sys- 
tem 


Probability of suc- 
cess for each task, 
technical appreach, 
subsystem 


QMDO requirements, 
tunds, other re- 
sources 


















Probability of pre- 
dicted sales, mar- 


gins, research needs, 
technical timing, and 
success 





None 


None 








In-house capability, 
external efforts, 
directed work 


Implicit--funding 
that gives reasonable 
probability of reach- 
ing technical goal on 
time 7 


















Probability of suc- 

cess of each task to 
reach technical ob- 

jective with funds, 

and in time 


Sponsored projects, 
in-house capability 
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Total budget 


Total budget 












Decision Algorithm 















Choosing pattern of 
development work to 
be done 


Not specific--DoD 
funding proportional 
to industry in some 
way 








AValue/asCost in de- 
scending order among 
project tasks 








APayoff/aCost equated 
over all technologies 











Linear program 









Decrease budget re- 
quired by eliminating 
low A"Exp. Value!?/ ° 
ACost among technical 
approaches 





Search possible 
shiitcs of funds among 


projects for mix 
giving maximum pdro- 
Paes 





Funds distributed 
down list of tasks 
orderea by priorities 


None 


Qualitative analysis 
of shifts in emprasis 
warranted from prior- 
ities, relevance, 
support 





Rank projects by de- 
scending exp. value/ 
cost 








a large degree, is directly correlated with the magnitude of 
the resources committed, which certainly should not come as 


any surprise. 


me LN RETROSPECT 

And so some general conclusions and statements can be 
gleaned from a review of these predictive models. Firstly, 
the worth-assessment process has not really been adequately 
structured by the models for the specific goals and purposes 
mee che military R&D system, particularly the EDP. To a 
large extent, this has been due to a generation of philoso- 
phies which restrict the options for dealing with future 
world situations by proposing concepts that are extensions of 
@ae current force structure and generally use the current 
Mechnnological approach to accomplish mission tasks. Conse- 
quently, and as previously indicated, R&D activities tend to 
focus on meeting system requirements rather than on gener- 
ating innovative ways to accomplish mission tasks. All too 
often the R&D activities in the EDP which are evaluated are 
not independent, as is necessary for worth-assessment proce- 
dures, but are often interrelated activities. Consequently, 
the projects selected usually do not provide all the technol- 
ogies necessary to develop and deploy a desired system. 

Secondly, the models exercised did not adequately provide 
procedures to assure the inputs necessary to meet the 
assumptions and requirements of weighting criteria. That is 
to say, missions, goals, and operational requirements were not 
(are not) being defined in terms that are worth-independent 
exef, 17). 
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Additionally, different worth criteria and underlying 


assessment processes are required to assess different activi- 


ties. Each model must therefore be specifically tailored to 


Seeoarticular activity. 


Wmen too, ehe cost effectiveness of 


actually using the models is very difficult to evaluate. The 


true effectiveness should generally be subjectively evaluated 


in terms of how useful an R&D manager finds the results. 


The future of R&D 


planning models similar to those identi- 


mrea thus far is not particularly promising in light of the 


observed methodological problems and managerial resistance to 


their use. The models just cannot, or have not been able to 


capture all the relevant and subtle aspects of the R&D and EDP 


planning process. The methodological problems preclude the 


use of such models even as decision information systems to aid 


Managers, Since their 

But organizations 
R&D must still select 
ties, however. These 


understanding of both 


output is highly questionable. 

performing Exploratory Development within 
and evaluate among many possible activi- 
decisions can be improved by a better 


the process of innovation and the Explor- 


atory Development planning process. It is important that 


organizations have systematic procedures and logical organi- 


zational structures to assure that the major planning tasks are 


effectively accomplished. Figure 6 has been suggested as a 


representative outline of the necessary elements of a contempo- 


rary planning process 


(Ref. 17). To develop such procedures 


and structures, the organization must clearly define what it 


hopes to accomplish by performing the exploratory development 
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OBJECTIVES 


IDENTIFICATION AND RANKING 
OF OPERATIONAL PROBLEMS 


OUTPUT: NEW OPERATIONAL CAPABILITIES 


CAPABILITIES , OBJECTIVES, OF 
NEEDS, PROGRAMMED 


FORCES 


LONG RANGE 
OPERATIONAL 


’ GOALS 





EVALUATION 


EVALUATION OF EACH 
CANDIDATE AS A SOLUTION 
TO IDENTIFIED PROBLEMS 


OUTPUTS LIST OF PROJECTS 
TECHNOLOGICAL MATCHED TO RANKED MENU OF R&D 


GORECASIS OPERATIONAL PROBLEMS PROJECTS 





DECISION 


SELECTION OF PREFERRED 
PROJECTS AND ALLOCATION OF 
FUNOS AMONG THOSE PROJECTS 


ESTIMATED OUTPUTS THE R&D BUDGET DECISION 


COSTS FOR THE NEXT PERIOO CRITERIA 





OTHER 
BUDGET CONSTRAINTS 


AND GOALS 





Figure 6 


Conceptual outline of a long range research 
development planning process. 
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within R&D. No technique, model or methodology is likely 
to improve the effectiveness of a poorly or inadequately 


implemented planning process. 
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V. DETERMINING ROI IN THE EXPLORATORY DEVELOPMENT PROGRAM 


It is difficult to argue with the fundamental proposition 
that any large enterprise ought to seek evidence of the effec- 
tiveness of existing programs and proposed program alterna- 
tives aS a basic guide to decision making. Yet few sound 
evaluation schemes and even fewer rigorous field experiments 
have actually been undertaken. Serious questions exist con- 
cerning the capacity of the government both to develop and 
use evaluative results. Much of the concern deals with the 
short supply of competent policy-oriented researchers both 
within and outside of the government, the inadequacy of avail- 
able methods and concepts for carrying out evaluative studies, 
and the shortage of organizations with the capacity to under- 
take large-scale evaluative activities. 

An evaluation method based on the tools of social research 
has been applied successfully to social action programs to 
determine program effectiveness. The basic principles of this 
methodology, while developed originally for social and educa- 
tion programs, actually are oriented toward any action program 
which can attest to a sound platform of goals and objectives. 
Thus, there appears to be a potentially viable technique 
which would be applicable to programs exhibiting outputs which 
present themselves in a somewhat ungquantifiable manner. 

This section presents an overview of the methodology, 
called Evaluation Research (ER), and attempts to show how it 


Might be applied to programs within the Exploratory Development 
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Program. While a complete treatise and development of an 
evaluation study on an exploratory program is beyond the 
scope of this paper, it is felt that the ER technique could 
be applied in particular applications to yield realistic 
measures of effectiveness of technology programs. A presen- 
tation of the background for the basic elements of such a 


study follows. 


A. EVALUATION RESEARCH FOR THE EDP 

Evaluation is a rather elastic word but common usage 
would generally imply the notion of judging merit. Evalua- 
fen has frequently had explicitly political overtones. It 
is designed to yield conclusions about the worth of programs 
and, in so doing, is intended to affect the allocation of 
resources. 

A form of evaluation, called Evaluation Research, has 
been devised and used in the past on social programs, although 
in its strictest sense is concerned with finding out how well 
action programs work. It represents the application of social 
Sclence research methods to discover information of importance 
to program practice and public policy. The tools of research 
are applied to make the judging process more accurate and 
objective. The evaluation thus establishes clear and specific 
criteria for success. 

The research process is particularly important when (1) 
the outcomes to be evaluated are complex, hard to observe, 
made up of many elements reacting in diverse ways; (2) the 


decisions that will follow are important and expensive; and 


62 





(3) evidence is needed to convince other people about the 
malidity OL the conclusions (Ref. 18). Clearly then, this 
scheme appears to have commonality with the R&D process in 
general and the Exploratory Development Program in particular. 

The rationale of Evaluation Research is that it provides 
evidence on which to base decisions about maintaining, 
insituationalizing and expanding successful programs and 
modifying or abandoning unsuccessful ones. For example, the 
methodology of determining program selection at a laboratory 
Block Program Office utilizes a decision matrix, as shown in 
Table V, provides much of the necessary basic information 
for initiating an evaluation research technique. The matrix 
is utilized as a tool for determining which programs should 
be supported and continued, as they relate to mission needs. 
Used in this context, evaluation research is increasingly 
important as a source of knowledge and direction. It can be 
used to tell which programs work and which do not, and points 
the way to better formulation of policy and program. It can 
be even more helpful as it proceeds to identify the effects 
of specific strategies and components within programs and 
separates out those that contribute to favorable outcomes 
from those that are ineffective or counter productive (Ref. 
eS) . 

If evaluation and cost-benefit analysis are to have any 
effect on decisions, there has to be some way to move from 
the data to the decision--some system for transmitting 


information and feeding it into the decision process. The 
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Block Program Decision Matrix 


Table V. 
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Planning, Programming, Budgeting System (PPBS) recently in 

use in DoD is such a system, but it starts with the definition 

of objectives rather than the delineation of goals. Providing 

a systematic approach to governmental planning, the distinctive 
characteristics of PPBS are: 

1. It identifies and defines fundamental objectives and 
considers all activities in terms of those objectives. 

2. It explicitly and systematically identifies alterna- 
tive ways of carrying out the objectives. 

3. It estimates the total cost implications of each 
alternative, looking not only at the current year but also 
Picure years. 

4. It estimates the expected benefits of each alternative. 

5. It presents the resulting cost and benefit comparisons 
for each alternative, along with identification of major 
assumptions and uncertainties. 

PPBS is more concerned with future options than with past 
experience, especially when applied in the technology program 
area. But in estimating expected benefits, it has to derive 
its figures from somewhere, and this is where evaluation 


Smould come in. 


B. PURPOSE OF EVALUATION RESEARCH 

Evaluation research is viewed by many as a way to increase 
the rationality of decision making and policy making. With 
objective information on the outcomes of programs, wise 
decisions can be made on budget allocations and program 


planning. Programs that yield good results will be expanded; 
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those that make poor showings will be abandoned or drasti- 
cally modified. 

An evaluation study does not generally come up with final 
and unequivocal findings about the worth of a program. [Its 
results often show small, ambiguous changes, minor effects, 
outcomes influenced by the specific events of the place and 
the moment. It may require continued study over time and 
across projects to speak with confidence about success and 
failure. 

For decision makers, evaluation evidence of outcome is 
only one input out of many. They must consider a host of 
Peder factors, from public receptivity to participant reac- 
mon, COStsS, availability of staff and facilities, and 
possible alternatives. Evaluation will definitely not take 
politics out of the decision making process. 

The purpose of evaluation research is to measure the 
effects of a program against the goals it set out to accom- 
plish as a means of contributing to subsequent decision 
making about the program and improving future programming 
(Ref. 19). The measurement of the effects refers to util- 
izing a research methodology to investigate outcomes of the 
program rather than efficiency or adherence to standards. 
The comparison of effects with goals stresses the use of 
explicit criteria for judging how well the program is doing. 

Evaluation research is committed to the principle of 
utility. If it is not going to have any effect on the 


decision, then it is clearly an exercise in futility. Weiss 
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(Ref. 


19) suggests that there are probably four kinds of 


circumstances in which evaluation is probably not worth doing: 


digs 


When there are no questions about the program. It 
goes on, and decisions about its future either do 
not come up or have already been made. 


When the program has no clear orientation. Program 
staff improvise activities from day to day, based 
on little thought and less principle, and the pro- 
gram shifts and changes, wanders around and seeks 
Gareect ion. 


When people who should know cannot agree on what the 
program is trying to achieve. If there are vast 
discrepancies in perceived goals, evaluation has no 
ground to stand on. 


When there is not enough money or staff sufficiently 
qualified to conduct the evaluation. Evaluation is 
a demanding business, requiring time, money, 
imagination, tenacity and skill. 


The all-purpose evaluation is a myth. But there are several 


types of uses for evaluation. Evaluation can investigate the 


extent of program success so that decisions such as these can 


be made: 


alle 


To continue or discontinue the program. 
To improve its practices or procedures. 


To add or drop specific program strategies and 
techniques. 


To institute similar program elsewhere. 
To allocate resources among competing programs. 


To accept or reject a program approach or theory. 


ELEMENTS OF AN EVALUATION RESEARCH PROCESS 


In abstract, the basic elements of the evaluation research 


process include (1) finding out the program's goals, (2) trans- 


lating the goals into measurable indicators of goal achievement, 
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(3) collecting data on the indicators of the participating 
elements (and for an equivalent control group), and (4) 
comparing the data on participants (and control elements) 
with the goal criteria. 

But this process in theory becomes much more complex in 
practice. A host of unexpected problems invariably arise 
such as: 

1. Program goals are often hazy, ambiguous, hard to pin 
down. 

2. Programs not only move toward official goals, but they 
also accomplish other things, sometimes in addition and some- 
times instead. 

3. The program 1S a congeries of activities, people and 
structures, some irrevelant. 

4. The evaluation question as posed ignores the issue of 
why the program succeeds or fails. 

Ideally then Weiss (Ref. 18) suggests the folowing as 
the necessary core issues to comprise the effective evaluation 
research process: 


i. Formulate the program goals that the evaluation will 
use aS criteria. 


2. Choose among multiple goals. 

3. Investigate unanticipated consequences. 

4. Measure outcomes. 

5. Specify what the program is. 

6. Measure program inputs and intervening processes. 


7. Collect and compare necessary data. 
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In its rudimentary form then, the evaluation process 
encompasses the interpretation and implementation, including 
the elements of policy, goals, theory, models, experiment 
(to check theory and provide criteria for success-failure). 

The experimental model remains the ideal in evaluation 
methodology, with random assignment of subjects to an experi- 
mental group which is exposed to the program stimulus and to 
a control group which is not, thus providing the research 
rationale. 

It collects before and after measures over a full cycle 
of the program and uses specified goal criteria. Its results, 
then, disclose the extent to which a consistent program has 
reached its stated goals. However, it rarely distinguishes 
why the observed results occur, what processes intervene 
between input and outcome or what the particular implications 
are for improving the effectiveness of the program. 

The application to the EDP, however, might just provide 
the necessary means with which to conduct a satisfactory 
selection/evaluation process, ultimately yielding a rational 
method for determining program worth. The stage is set; what 
remains now is a progresSive-oriented faction of the Explora- 
tory Development Programs, willing to commit the necessary 
resources of time, personnel and money to explore the poten- 


tial of this new technique. 
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VI. SUMMARY AND CONCLUSIONS 


The Navy Exploratory Development Program is currently 
undergoing major revision to align its management planning 
system with the philosophies of the new wave of thinking 
regarding program effectiveness; i.e., to meet the Navy's 
Baere Obligation £0 put government resources to their most 
effective use in mission accomplishment. 

Traditionally, technology has gone on to develop subsystems 
and systems concepts independently of specific mission needs. 
Further, rather than actively building a strong technology 
base through innovation, old technologies have been extended 
to improve performance of old kinds of products. New systems, 
while becoming more complex and expensive have actually become 
less technologically advanced. 

It is recognized that it 1s imperative that cost-benefits 
be maximized on technology base programs. This, though, can 
only be realized when the base programs are driven from oper- 
ational needs and mission requirements. 

A complete and sound understanding of the functions and 
underlying philosophies and policies of the technology pro- 
gram 1s essential to determine that program's worth, or 
return-on-investment. 

sound foundations must be established from which goals and 
strategies can be derived to provide meaningful inputs for 
evaluation methodologies of contemporary exploitation, rather 


than basing output measures on dollar values. 
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In terms of the policy platform for the Exploratory Devel- 
opment Program, the underlying principles evidenced in the 
new rationale include: 

e Supportive of technological advances being of funda- 

mental importance to operational needs. 

@ Quest for technical and managerial excellence at all 

levels. 

@ Recognition of the necessity for both top-down direction 

and bottom-up inputs. 

e Accountability essential at all levels. 

e Necessity for strategies as tools to implement policies. 

@e Recognition that sometimes technical advance must be 

injected into the main stream of the system forcibly. 

@ Recognition of essential need for adequate information 

systems to allow interaction. 

With a strong push in Navy policy to promote accountability, 
a more decentralized organization is developing, as evidenced 
by the formation of the Block Program Offices. This allows 
a means for providing technical management in the laboratories 
and Systems Commands with the necessary authority to plan and 
manage technical efforts related to the pursuit of significant 
objectives. 

As a result of direction by the ASN (R&D) to establish and 
implement a program to improve the exploratory development sys- 
tem, to improve higher echelons' understanding of the system 
and to justify 6.2 expenditures, the Exploratory Development 


Program has been, and currently is being radically revamped. 
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Central to these modifications is the use of dynamic “technical 
strategies," designed to focus exploratory development more 
directly and visibly on operational needs. The primary objec- 
tives of the new management system are to: 

e Provide top-down direction by the use of technical 
strategies, prioritized technical objectives and 
definition of major program thrusts designed to meet 
operational requirements. 

e Provide visibility by relating the program to Navy 
functions and operational needs. 

® Promote innovation and concentration on highest leverage 
opportunities. 

@e Expedite transfer of projects to advanced development 
(6.3) and use of new technical capabilities developed 
in research (6.1). 

e Demonstrate program relevance and cost-effectiveness. 

It is essential that a firm understanding of the technical 
strategies, functional areas and program elements be had to 
augment an understanding of the EDP objectives and transition 
steps. Only through a thorough knowledge of the inner work- 
ings of the new system can any meaningful review, evaluation 
and assessment methodology be realized. While no established 
program exists currently, there certainly is potential for 
developing a viable methodology, particularly with the well 
structured and goal oriented EDP being implemented today. 

Worth-assessment processes to date have not been able to 


be adequately applied to the R&D structure in general, nor the 
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Pim aticiwtare tits 1s Gue primarily to the traditional 
programs being system requirement rather than mission need 
(goal) oriented. Further, earlier models did not define in- 
puts in terms that were worth-dependent because different 
worth criteria and underlying assessment processes must be 
mecec tically tailored to particular activities. 

The future of R&D planning models is not Fee dtanlaeaie 
promising in light of observed methodological problems and 
managerial resistance to their use. The models just have not 
been able to capture all of the relevant and subtle aspects 
of the R&D and the EDP planning process. 

But organizations performing exploratory development must 
Still select and evaluate programs some way. These decisions 
can be improved through a better understanding of both the 
process of innovation and the exploratory development planning 
process. 

New techniques and methodologies, as well as revisions of 
old ones, are constantly being identified, many receiving par- 
ticular area utilization, which will contribute to the process 
of rational decision making; i.e., program evaluation and 
gosessment. One method felt to have high potential in the 
exploratory development area is that called Evaluation Research. 
Originally developed for application to social action programs, 
it is best utilized in situations where sound platforms of 
goals and objectives can be established. Thus, there appears 
to be a potentially viable technique which might be applied to 
programs in the EPD which yield outputs in a somewhat unquanti- 


EPaple Manner. 
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The stage has been set for the design of a new methodo- 
logical tool to assess the worth of programs in the EDP. 

A technique has been identified which has high potential. It 
is strongly recommended that further study be undertaken, 
using the background and management planning systems presented 
in this thesis, to ultimately demonstrate the ability of eval- 
uation research to effectively determine a measure of worth 

of programs in the Exploratory Development Program. 

The evidence suggests that such a tool can be realized. 
What is required now is to uncover a progressively oriented 
faction of the EDP willing to commit the necessary resources 
to more fully explore and demonstrate the potential of this 


research technique. 
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APPENDIX A 


EXPLORATORY DEVELOPMENT PROGRAM RESPONSIBILITIES 


The following delineates Exploratory Development Program 
management responsibilities. It discusses basic organization 
design principles, the responsiblities of Program organiza- 
tions and officials, and the various roles played by non- 
NAVMAT organizations and officials in the 6.2 Program. 

Organization Design Concepts. The Exploratory Development 
Program management system is designed to strike the best com- 
promise between the need to administer public resources in 
accordance with applicable laws, regulations, and policies, 
and the need to provide technical management with the freedom 
required to hold them accountable for results. The responsi- 
bilities described for the Exploratory Development Program are 
based on the following design principles: 

MescemeccouMecoN tive Establishing the conditions neces— 
ary to hold individual officials (as distinguished from diffuse 
bureaucracies) accountable is a central consideration. Pre- 
requisites to accountability include: 

a. Assigning responsibility for results to an individual. 

b. Provide that individual with the necessary authority 
and resources to accomplish the required results. If a mana- 
ger is to be held accountable, he must have the opportunity 
to pursue objectives by the means which he judges most 


effective. 
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c. Providing the responsible individual with the oppor- 
tunity to perform by limiting the demands on his time. This 
includes demands by superiors and their staffs, and by other 


external officials and organizations. 


elear separation, of sresponsibilities. Responsibilities 


for determing the objectives to be pursued, and the technical 
effort required to fulfill those objectives, have been clearly 
and deliberately separated. Although it is essential that 
spokesmen for these two interests conduct an in-depth dia- 
logue in the process of defining the Program, it 1S important 
to avoid conflict of interest. In the management system 
described herein, technical people are not in a position to 
dictate "requirements," and management people and sponsors 

are not in a position to control the selection of technical 


memrcoach. 


Quality information for management. The management sys- 
tem has been designed to provide the ASN (R&D), CND, and 
lower-level officials with all the information they require 
to implement their Program responsibilities. The interaction 
of organizations generates quantitative information required 
for technical management to put technical resources to their 
most productive use, and for higher-level management to 
reallocate resources as required to increase overall Program 
output. A system of institutionalized Program Reviews ensures 


the quality of information provided to top management. 
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Management by objectives and results. The management 
system relies on mutually established and agreed-upon objec- 
tives and goals rather than on management by specific direc- 
tives, detailed procedures, elaborate reporting systems, 
frequent reviews, and other close controls. Management is 
by "self-control" in pursuit of the most efficient accomplish- 
ments of stated objectives. Higher-level review primarily 


measures results achieved in relation to approved objectives. 


"People information system" for coupling. The process of 
defining EDP projects which collectively support Department 
of the Navy purposes requires extensive and intimate inter- 
action between competent spokesmen for both sponsor and per- 
former organizations. The interaction of Technical Strategists 
with spokesmen for Mission Needs and with technical managers 
in the laboratories is a vital element of the "people infor- 
mation system." This face-to-face, paper-free information 
system provides a mechanism for the flow of information on 
needs and operational environment to the technical community, 
and for the reverse flow of knowledge about technical oppor- 


Pinmuty to the CNO/CMC staffs. 


Program Management Responsibilities 

Deputy Chief of Naval Material (Development) (DNM(D)/ 
Chief of Naval Development (CND)). As DCNM(D), this officer 
reports directly to CNM. As CND, he is directly responsible 
to ASN (R&D). DCNM(D)/CND has overall control of the Explor- 


atory Development Program and its funds. He carries out this 
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general responsibility primarily through the DNT. His func- 
tions include: 

a. Reviewing and approving the Exploratory Development 
Program Plan (EDPP) at each stage in its evolution (1.e., the 
Draft and Tentative EDPPs, as well as the EDPP. 

b. Appointing Technical Strategists. 

c. Approving Technical Strategies. 

Gi. Approving Program Plans and related budgets. 


e. Directing the allocation of funds. 


Dieeceton or Technology Programs/Director of Navy Technol- 
ogy DTP/DNT. As DNT, this official is directly responsible 
to the ASN (R&D). As DTP, he is responsible to the DCNM(D)/ 
Sip. Acting for the CND, he is responsible for the planning, 
execution, and evaluation of the Exploratory Development Pro- 
mem. His functions include the following: 

a. DeSigning a balanced and integrated overall Explora- 
tory Development Program which maximizes the contribution of 
the Technology Base to the mission capability of Department 
of the Navy operating forces. 

b. Developing and documenting the Exploratory Development 
Program Plan (EDPP) for review and approval by CND and ASN (R&D). 

c. Nominating Technical Strategists and appointing 
Strategy Team members. 

d. Reviewing and coordinating Technical Strategies. 

e. Developing a Program-wide priority list for concept- 


related technical effort. 
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f. Negotiating Program Plans (and related budgets) with 
Systems Command Directors of Research and Technology (SYSCOM 
03s) and with Technical Directors of the laboratories (Lab 
TDs), and recommending those plans to CND for approval. 

g. Appointing Program Review Team leaders, and assisting 
in the staffing of Review Teams. 

h. Scheduling and chartering External Program Review. 

i. Conducting Program Reviews at the SYSCOM 03/Lab TD 


level. 


SYSCOM Directors of Research and Technology (SYSCOM Q3s). 
The SYSCOM 03s plan and execute that portion of the overall 
Exploratory Development Program related to System Concepts 
(SCs) and other products within the material responsibility 
areas of their SYSCOM. The general responsibilities of a 
SYSCOM 03 include: 

a. Designing a balanced and integrated overall SYSCOM 
Exploratory Development Program which supports Objectives in 
the approved Mission Area Technical Strategies. 

b. Documenting planned technical efforts in a Program 
Plan for negotiation with and approval by the DNT. 

c. Prioritizing concept-related objectives set forth in 
eae rogram Plan. 

d. Providing designated personnel to serve as Technical 
Strategists and as members of Strategy and Program Review 
Teams, as requested by DNT. 


e. Negotiating and approving proposed TAPs. 
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f. Reviewing proposed changes in TAPS approving desired 
changes within his authority, and submitting desired changes 
beyond his authority to DNT for appropriate action. 

g. Monitoring execution of his SYSCOM's technical program, 
and taking actions as appropriate. 

h. Ensuring the Internal Review of his SYSCOM's program, 


and providing copies of Internal Review Reports to DNT. 


Laboratory Technical Directors (Lab TDs). The Lab TDs plan 
and execute that portion of the overall Exploratory Develop- 
ment Program related to the development and utilization of 
technology within the areas of Laboratory responsibilities. 
The general responsibilities of the Lab TD include the follow- 
mie LuUnctlons: 

a. Designing a balanced and integrated overall Laboratory 
Exploratory Development Program which supports approved Tech- 
nology Area Strategies. 

b. Documenting planned technical efforts in a Program 
Beem LOr negotiation with and approval by the DNT. 

Cc. Prioritizing technology-related objectives set forth 
in the Program Plan. 

d. Providing designated personnel to serve as Technical 
Strategists and as members of Strategy and Program Review 
Teams, as requested by DNT. 

e. Negotiating and approving proposed TAPs. 

£. Monitoring execution of his Laboratory's Technical 


Program, and taking actions as appropriate. 
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g. Reviewing changes in proposed TAPs approving desired 
changes within his authority, and submitting desired changes 
beyond his authority to the DNT or SYSCOM 03 as appropriate. 

h. Ensuring the Internal Review of his Laboratory's pro- 


gram, and providing copies of Internal Review Reports to DNT. 


Responsibilities for preparation of Technical Strategies. 
Each Technical Strategy is developed and/or updated by a Tech- 
nical Strategist and a supporting Technical Strategy Team. 

Technical Strategists. A Technical Strategist is respon- 
sible for the development and updating of each Technical 
Strategy. Strategists report to DNT for all matters relating 
to their Technical Strategies. Technical Strategists may be 
recruited from any Department of the Navy organization; 
normally, however, they will be NAVMAT, Laboratory, or Systems 
Sonmmand officials "double hatted" to CND for this duty. The 
general responsibilities of Technical Strategists include: 

a. Identifying Strategy Team needs with the appropriate 
knowledge and organizational affiliations. 

b. Developing and/or updating the assigned Technical 
Strategy as required (with the aid of the Strategy Team). 

c. Maintaining current knowledge of the Technical Strategy 
in general (and its MA or TA Objectives in particular) and of 
eel relevant technical capabilities, current efforts, and 
planned efforts. 

d. Reviewing all TAPs which purport to support Objectives 
fisted in the Strategy, and taking appropriate actions to im- 
prove the effectiveness of technical contributions to higher- 
level objectives. 
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e. Serving as person-to-person channels between CND/DNT, 
Strategy Teams, and the technical community for the flow of 
needs information related to their Technical Strategies. 

This function includes helping technical personnel establish 
direct contact with individuals most knowledgeable about issues 
of interest. 

Strategy Team. Strategy Teams are constituted so that 
they have within the Team the essential knowledge and expertise 
required for the development of the Technical Strategy--without 
reference to documents except for detailed information. Such 
representation, in addition to helping in the cost-effective 
preparation of an adequate Strategy document, is an essential 
element in the development and maintenance of the "people 
information system." Ideally, the Strategy Team includes 
members from organizations which both require information on 
the Strategy, and which are sources of information required in 
the development of the Strategy. Each Strategy Team should 
include individuals who served on the Team in prior years; 
this ensures the preservation of a paper-free institutional 
memory. MA Strategy Teams include members from all TA Strategy 
Teams for technologies critical to the attainment of the MA 
Objectives. Participation of these people provides a direct 
and paper-free injection of information on technological capa- 
bilities and potential capabilities into the development and 
updating of the Strategy, and a reverse flow of Mission Need 
information into the program planning and execution activities 


of their parent Systems Commands and Laboratories. TA Strategy 
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Teams include spokesmen for SCs which use subject technology, 
members from organizations developing subject technology, and 
representatives of the research community involved in develop- 


ing the base of knowledge underlying the subject technology. 


Source of Strategy Team members. Strategy Team members 
are recruited from both Government and the private sector. 
Department of the Navy technical organizations are directed 
to provide members to Technical Strategists on a negotiated 
basis. Assistance in recruiting non-Department of the Navy 
personnel will be provided by DNT (Attention MAT 03T3). 
Strategy Team members from Department of the Navy organiza- 
tions are to be provided travel and analytical support by 
their parent organizations as required. 

Program Review Teams. There are two kinds of Program 
Review: Internal Review and External Review (see Chap. IV). 
Internal Review Teams, under the leadership of the cognizant 
Laboratory Technical Director or SYSCOM 03, conduct self- 
reviews of all detailed technical programs. External Review 
Teams, under the leadership of a designated member of DNT's 
Staff, conduct External Program Reviews at the Laboratory/ 
SYSCOM level. 

Staffing the Interval Reviews. SYSCOM 03s and Laboratory 
Technical Directors are responsible for determining the size 
and make-up of each Laboratory or SYSCOM Internal Review 
Team, and for asSigning individuals from within their respec- 
tive organizations to serve as Team members. TD/SYSCOM 03s 
also have the option of recruiting Team members from outside 


their organizations. 
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Staffing External Reviews. Two primary considerations 
determine the staffing of External Review Teams: (1) Team 
members should be qualified to verify the validity of Internal 
Review information without reference to additional documenta- 
tion, and (2) the External Review Process should establish 
a "people information system" which contributes to the effec- 
tive planning and execution of the overall Program. The DNT- 
chartered External Program Review Team is a broad-based ad hoc 
group. DNT appoints an External Review Team leader from his 
staff; he also appoints other Team members from among inter- 
ested and qualified Navy and non-Navy organizations. External 
Review Team members include: (1) current and/or former members 
of relevant Strategy Teams, (2) DNT staff members, (3) members 
of previous DNT Program Review Teams (to ensure the preserva- 
tion of a paper-free institutional memory; these individuals 
can note what has been done about deficiencies discovered in 
previous External Program Reviews), (4) representatives of 
organizations which supply inputs to the programs under review 
(e.g., ONR Scientific Officers for disciplines critical to the 
technologies involved), (5) representatives of organizations 
which use the outputs of the programs under review, (6) tech- 
nology managers who conduct similar programs, and (7) experts 
from the wider technical community, including industry. Extern- 
nal Program Review Team members are important links in the 
paper-free “people information system." 

Biteocadee. the DNT staff provides the DNT with supporting 


services, information, and counsel required for effective 
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performance of DNT's Exploratory Development Program manage- 
ment functions. Individual staff members are assigned cogni- 
zance over specific matters--such as individual Technical 
Strategies, Laboratories and SYSCOMs and their related Pro- 
gram Plans, and Program Elements. Specific duties of these 


cognizant staff members are discussed below. 


Technical Strategies. Duties of the DNT staff members 
assigned cognizance over a Technical Strategy include: 

a. Achieving and maintaining in-depth knowledge of the 
Technical Strategy, its rationale, and of all on-going or 
planned technical effort supporting Objectives in the approved 
Secactegy . 

b. Serving as a member of the Strategy Team for that 
Bema Clegy . 

c. Providing staff services for the recruitment and 
assignment of Strategy Team members, in compliance with staffing 
requirements developed by the Technical Strategist. 

ad. Alerting the DNT to developments or trends related to 


the Strategy which require DNT's attention. 


Organization and Program Plans. Duties of DNT staff mem- 
bers assigned cognizance over technical organizations and 
related Program Plans include: 

a. Achieving and maintaining in-depth knowledge of the 
planning and execution aspects of the organization's technical 
program--including all changes to the originally approved 


program. 
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b. Reviewing proposed Program Plans and providing DNT 
with appropriate information and counsel. 

c. Serving on the Program Review Team (as leader or 
member) . 

d. Determining requirements for Review Team personnel, 
and taking necessary actions to facilitate the recruitment 
and assignment of appropriate personnel. 

e. Alerting the DNT to developments or trends related 
to the organization (or its Technical program) which require 
Bye's attention. 

Peo Louvlaing Obal and written information about the 
organization and its program to Government officials and pri- 


mace sector individuals with a need to know. 


Program Elements. Duties of DNT staff members assigned 
cognizance over a Program Element include: 

a. Achieving and maintaining in-depth knowledge of: 
(1) planning and execution aspects of work included within the 
Program Element, (2) technical objectives and supporting 
rationaies for the work performed, and (3) all actions and 
femeemes Of higher level officials, including the Congress, 
related to the Program Element. 

Db. Preparing Program Element Descriptions (PEDs) and any 
other documentation required in connection with the annual 
budget submission, apportionment process, Congressional 


hearings, etc. 
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c. Preparing PE-level TAP Aggregates using TAP informa- 
tion approved by SYSCOM 03s and Lab TDs (and by DC/S RD&S and 
CG MCDEC for Marine Corps programs). 

d. Alerting DNT to developments or trends related to the 
Program Element which require DNT's attention. 

e. Providing information, both orally and in writing, 
concerning the Program Element to Government officials and 


private sector individuals with a need to know. 


Pee -aelagmhocPens DLlit1es- | Other organizations and 


individuals outside of the Naval Material Command play signifi- 
cant roles in the Exploratory Development Program. 

Congress. The Congress provides the funds for the 6.2 
Program and establishes funding controls at the Program Ele- 
ment level. Congress may, on occasion, control the funding of 
individual projects. Management responsiveness to Congress, 
credibility of information, and utility of results are essen- 
feel te continued stability of the Navy's Program. 

Director of Defense Research and Engineering (DDR&E). The 
DDR&E exerts influence at all levels of the Program. He can 
compare the management and technical content of the Exploratory 
Development Programs of all Services, and can direct all tech- 
nical programs in an area to be assigned to one Service. The 
DDR&E is the dominant DoD agency in the justification of work 
and the budgeting of funds. 

Assistant Secretary of the Navy (Research and Development 
(ASN(R&D)). The ASN (R&D) is responsible for the RDT&E (N) 


appropriation, and for the planning and execution of the entire 
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Department of the Navy RDT&E Program (including Exploratory 
Development). ASN (R&D) reviews and approves the various 
versions of the EDPP on an exception basis. He is also charged 
with justification of the Navy EDP to DDR&E and Congress. The 
small ASN (R&D) staff relies heavily on the CND and his staff 
for information and presentation of the program. Close work- 
ing relationships and mutual confidence between CND/DNT and 


the ASN (R&D) are essential. 


Chief of Naval Operations/Commandant of the Marine Corps 
(CNO/CMC). The CNO and CMC influence the 6.2 Program by estab- 


lishing operational needs and by directly sponsoring Advanced 
and Engineering Development. As the primary markets for the 
Program output, the CNO/CMC must be satisfied with its mili- 
tary relevance. The CND, assisted by the SYSCOMs, interprets 
the operational needs described by CNO/CMC in the Technical 
Strategy documents. The subsequent discussion, understanding, 
refinement, and definition of operational needs lead to devel- 
opment and acceptance of a useful product. CNO/CMC review 
all SCs when they are first submitted, and annually as long 
as they are included in the Program. CNO/CMC estimate the 
military worths of all SCs and assign them quantitative mili- 
Beeeworth ratings, "Vy" ratings, for use in planning Technical 
Strategies and supporting technical programs. 

Chief of Naval Research (CNR). In addition to his respon- 
sibilities for the Naval Research Program, the CNR is responsi- 


ble for sponsoring these exploratory development projects 
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which are necessary to ensure orderly transition from research 
to development. For the projects which he sponsors, CNR func- 
tions like a Systems Command. 

Chief of Naval Personnel (CNP) and Chief, Bureau of 
Medicine and Surgery (CHBUMED). CNP/CHBUMED are the primary 
sponsors for Exploratory Development relative to the human 
subsystem and to medical and dental systems. They are under- 
stood throughout this instruction to function like a Systems 
Command in these areas. 

inaustry/Academia. Industry and educational institutions 
interact principally at the SYSCOM and Laboratory levels. 

They provide Laboratories and SYSCOMs with assistance, on a 


contract basis, in developing both technologies and products. 
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APPENDIX B 


EXPLORATORY DEVELOPMENT PROGRAM DOCUMENTATION 


The documentation for planning and justifying the Explora- 
tory Development Program is described in this appendix. The 
basic document through which the Chief of Naval Development 
(CND) and the Director of Navy Technology (DNT) control the 
Program is the Exploratory Development Program Plan (EDPP). 
The EDPP is supported by Technical Strategy documents, Labor- 
atory and System Command Program Plans, and Systems Concepts 


SCs). 


BRouvonaeOm. Development Program Plan (EDPP). The EDPP 


Momieac authoritative statement of the content and rationale 

of the Navy's Exploratory Development Program. It defines 

the major thrusts of the overall Program, identifies major 

Program goals, and apportions resources among all elements 

of the Program. Further, it summarizes and provides an inte- 

grated overview of the Technical Strategies and the supporting 

Laboratory and Systems Command Program Plans. The EDPP is 

prepared by the Director of Navy Technology (DNT) on the basis 

of: (1) budget and program guidance from higher authority, 

(2) Technical Objectives developed by the Strategy Team, 

(3) Program Plans developed by Laboratories and SYSCOMs, and 

(4) Internal and External Program Review Reports. The EDPP 

is continually updated to reflect changes in these inputs. 
Technical Strategies. Technical Strategies serve two 


major purposes: They effectively relate the Exploratory 
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Development Program to Department of the Navy Mission Needs, 


and they make that relationship visible and credible. 


Functions of Technical Strategies. Technical Strategies 
have both direct and indirect functions. 

Direct Functions. Technical Strategies are designed to: 
(1) take maximum advantage of new technical opportunities, 

(2) exploit deficiencies in opposition capabilities, and 
(3) provide prompt response to perceived requirements for 
Superior naval capability. 

Indirect Functions. Strategies provide the vehicle for a 
two-way information flow on needs and technological capabili- 
ties (and possibilities). They also serve as the means for 
demonstrating the general rationality--and specifically the 
mission-relevance in output-oriented terms--of the Exploratory 
Development Program. 

Kinds of Technical Strategies. There are two kinds of 
Technical Strategies: those which relate to the Mission 
Areas for which the Navy has responsibility and those which 
relate to the Technology Areas which are basic to the Navy's 
operational capabilities. 

Mission Area (MA) Technical Strategies. MA Technical 
Strategies directly link the Exploratory Development Program 
to the major missions of Department of the Navy operating 
forces. MA Strategies are developed for assigned Mission 
Areas or for established segments of Mission Areas. Since 


MA Strategies are designed both to relate the Program to 


or 





Operational missions, and to facilitate understanding of that 
relationship by officials and organizations with influence 
over the Department of the Navy Program, MA Strategy Areas 
are structured to be compatible with the structure of Mission 


Areas used by DDR&E in the Mission Area Summaries (MAS). 


Technology Area (TA) Mec hnaca | Seaecgres. TA Technical 


Strategies set forth objectives for technology and the sup- 
porting rationales. The structure of areas for which TA 
Strategies are developed is also derived from the DDR&E mission 
Area Summaries. TA Technical Strategies so defined are also 
compatible with the Technology Coordinating Papers (TCPs) 


issued by the OSD. 


Preparation of Technical Strategies. Preparation of both 
MA and TA Technical Strategies involves the identification 
and evaluation of needs, and the identification of present, 
future, and potential capabilities for satisfying those needs. 
All Technical Strategy documents employ a similar format. 
Each describes the relevant Strategy Area (this will be either 
a Mission Area or a Technology Area), sets forth Objectives 
and the supporting rationales, identifies how Objectives in 
the Strategy Area are being pursued currently, and references 


sources of additional information. 


Mission Area (MA) Technical Strategies. In general, each 
MA Technical Strategy sets forth: (1) Mission Area Objectives 
for the Mission Area, (2) the supporting rationales for those 


Objectives, and (3) a discussion of present and future 
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exploratory development efforts relevant to MA Objectives. 
Preparing MA Technical Strategies requires a comprehensive 


understanding of operational needs. 


Mission Need Information. Mission Need information is 
available through both formal and informal systems. Informa- 
tion on operational needs is provided primarily through an 
informal face-to-face system based on and backed up by a 
document-based system. Information on needs is derived from 
a variety of sources including Fleet letters, CNO memoranda, 
extrapolation of threat statements, etc. Mission Need infor- 
mation flows through the Navy/Marine Corps organization to 
the Exploratory Development Program by means of three formal 
documents: the Navy Science and Technology Objectives (STOs), 
the Marine Corps General Operational Requirements (GORs), and 
the Operational Requirements (ORs). The first two documents 
are aimed primarily at guiding the inception of projects and 
programs (although guidance may be derived from them at any 
point in the process). The Operational Requirements (ORs) are 
concerned with the transition of System Concepts to Advanced 


Development. 


Mission Area Objectives. MA Objectives are objectives 
for operational capabilities within the Strategy Areas. They 
are set forth as means-free statements of needs for opera- 
tional capabilities required to execute the mission. These 
statements are prepared in a manner which satisfies the 


requirements for statements of Mission Needs for the initiation 
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of major system acquisitions as set forth in OMB Circular 
A-109 and implementing DOD directives and instructions. MA 
Objectives should be considered identical to the Mission 
Element Need Statements (MENS) defined in DOD Directive 
5000.1 and should conform to the format specified in Appendix 
D. All MA Objectives are to be numbered in a manner which 
facilitates the information capabilities discussed in 4.6. 
The rationale for each MA Objective should be explained 
clearly and concisely and should reference all official docu- 
mentation (such as STOs and GORs) on which it is based in 


whole or in part. 


Actual and Potential Gayo elssbibab ents) 6 Associated with each 


MA Objective is a discussion of means for satisfying the 
Objective. This discussion consists of a concise description 
of each system, System Concept, or other way (such as a combi- 
nation of forces) to satisfy the Objective, along with refer- 
ences to primary sources of additional information. Addition- 
ally, capabilities relevant to each Mission Need are to be 
identified and summarized briefly. These include present capa- 
bilities, systems in development or production but not yet 
operational, and Demonstrated-feasible, Presumed-feasible, 

and Pre-feasible System Concepts (SCs). All SCs will include 
military worth ratings by CNO and/or CMC. The listing of 
Demonstrated-feasible SCs will also identify concepts which 


were dropped since the last update of the Technical Strategy. 
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Technology Area (TA) Technical Strategies. Each TA 


Technical Strategy: (1) Describes current technical efforts 
and relates these to identified Mission Needs, (2) sets forth 
Objectives for technology and the supporting rationales of 
those Objectives, (3) justifies the importance of technical 
opportunities and potential advancements in output-oriented 
terms, and (4) explicitly summarizes and presents justifica- 
tions for present, future and potential technological capabil- 
ities by specifically referencing relevant SCs. TA Technical 
Strategies are developed and/or updated in a manner analogous 
to the process described above for MA Technical Strategies. 
The process involves defining Technology Needs, setting forth 
the rationales for those needs, and describing present, future, 
and potential capabilities relevant to identified Objectives. 
A Technology Area is concisely defined in terms of the tech- 
nologies included within its scope. Where further clarity 

is needed, a Technology Area may be further defined by identi- 


fying related technologies beyond its scope. 


mechnologveNeed information. Technology Need information 
is derived from three principal sources: (1) the systematic 
analysis of Pre-feasible SCs, (2) feedback from the develop- 
ment and manufacturing processes, and (3) feedback from the 
operating forces. Information from analysis of SCs takes the 
form of documented statements of technological problems or 
barriers; feedback information flows primarily through person- 
to-person links and is introduced into the Strategy delibera- 
tions most frequently through the personal experience of 
Strategy Team members. 
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Technology Area Objectives. Objectives for technological 


capabilities are set forth as statements of needs for tech- 
nological capabilities, termed Technology Area Objectives, 

or TA Objectives. These are statements of needs both for the 
development of new technological capabilities in support of 
SCs, and for the preservation of a base of technology under- 
lying present operational capabilities (where it is judged 
that such a technology base would be endangered in the absence 
@i action to preserve it). TA Objectives are numerically 
identified in a manner which facilitates the information capa- 
bilities discussed above and in 4.6. The rationale for each 
TA Objective should be set forth clearly and concisely and 
should reference all official documentation (such as techno- 
logical problem statements from analyses of SCs) on which it 
is based. Every rationale should present a convincing case 
which describes how the Objective is critical either to 
achievement of higher level of mission capability or to the 
preservation of capabilities which would be endangered in the 


absence of actions the Objective is designed to stimulate. 


Actual and Potential Capabilities. Technological capa- 
bilities relevant to each TA Objective are identified and 
summarized briefly, as are the rationales for each of those 
capabilities. These include statements and rationales for 
current capabilities, for capabilities anticipated on the 
basis of planned technical effort, and for capabilities which 


are potentially achievable if a specified level of technical 
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effort is assumed. The accompanying discussion identifies 

the leading organizations possessing, acquiring, or capable 

of acquiring the capabilities. Relevant ongoing or planned 
technical programs are identified, and their predicted impacts 


on capabilities are specified. 


PEogeatmn ao ee sOocater ans are Lhe primary instruments 
through which the DNT exerts top-down influence over portions 
of the Exploratory Development Program assigned to each SYS- 
COM and Laboratory. Program Plans document each SYSCOM's 
or Laboratory's integrated and balanced plan for maximizing 
the contribution of its own exploratory development efforts 
to Department of the Navy needs. Program Plans are not 
collections of Task Area Plans (TAPS). (TAPS are prepared 
Peeeeeeic Program Plans are approved and are designed to 
Carry out objectives listed in those Plans.) Analogous to a 
DCP for approval, funding, and management of an acquisition 
program, the Program Plan constitutes a "contract" between 
DNT and the SYSCOM 03/Lab TD for the Laboratory/SYSCOM por- 
tions of the 6.2 Program. The Program Plan is the basis for 
the negotiation, approval, funding, and subsequent review 
of each SYSCOM's and Laboratory's Exploratory Development 
Program. The content and format for Program Plans are sum- 


marized briefly below. 


pocennology Needs and Thrusts. This section will provide 


an introduction and overview of the proposed technical pro- 


gram, setting forth the major thrusts of the program and the 
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rationale for those thrusts. It will also discuss Significant 


changes from past years and the rationale for those changes. 


Program Objectives. The technological objectives the 
technical program is designed to achieve will be set forth, 
aS will the rationale for those objectives. These objectives 
are to be stated in quantitative terms (to the extent possible) 
to facilitate subsequent review of results. The rationale 
for objectives should explicitly identify the approved 
Technical Objectives which the program objectives are designed 


fo sSupport. 


Proposed Program. The technical program to achieve each 
program objective will be briefly described in both narrative 
and quantitative terms. This description should include the 
proposed dollars to be spent in support of each program objec- 
tive, the prospective performing institution(s), and major 


milestones scheduled for achievement during the program year. 


iMimrMOUSe7/eonerace Ratio, The program plan will specify 


the relative proportions of work to be performed by Govern- 
ment organizations (in-house) and by industry and other non- 
Government organizations (under contract). The DNT will 
negotiate the in-house/contract ratios with each SYSCOM and 
Laboratory to ensure that the overall Program conforms to the 


most recent policy guidance. 


ProgrameluEmover. “Ongoing efforts should be critically 
reviewed and sufficient work completed or terminated, to 


permit a minimum of 20% of the technical program to be composed 
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of new starts. The Program Plan will specify the percentage 
of the program which is new, and will identify efforts being 


completed or terminated. 


Priorities. The technical program will be organized into 
decision packages corresponding to discrete elements of the 
proposed program (such as all work related to specific System 
Concept or Laboratory subproject). These decision packages 
will be presented, along with cost information, ina rank 
ordering. This is intended to facilitate the elimination, or 
addition, of entire projects (rather than to make across-the- 
Beata Lunding reductions which could result in subcritical 
funding and the inefficient use of resources). Rank ordering 
also facilitates the phased initiation of new starts to accom- 


modate Congressional budget actions and deferrals of funds. 


System Concepts. Priority rankings for SYSCOMs will be 
Organized around the SCs which are the objectives of SYSCOM 
efforts. Information to be presented will include: 

a. A master rank-order of all ongoing and new starts of 
all three classes of SCs. 

b. A rank-order of both ongoing and new starts for each 
of the three classes of SCs. 

Cc. <A rank-order for proposed new starts for each category 


Of SC, 


Laboratory Subprojects. The proposed Laboratory program 


will be presented as a master rank-ordered list of subprojects 


and will include ongoing work and proposed new starts. 
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System Concests (SCs). An SC is a System Concept for 


satisfying one or more MA Objectives. SCs provide a mechanism 
for: (1) identifying technology needs, (2) focusing technical 
effort on the technological advances required to make these 
high-leverage system concepts feasible, and (3) aiding trans- 
fer of that technology to development once technical feasi- 
bility has been achieved. Although normally originated by 
Laboratories, Systems Commands, or industry, SCs may be origi- 
nated by any organization. Instructions for the documentation 


of SCs are contained in Appendix E. 


eClssseon olay Seem ConeSsrts- ses fall into three classes: (1) 


concepts presumed not yet technologically feasible without 
advances in underlying technology, (2) concepts presumed to 
be technologically feasible within five years but for which 
feasibility has yet to be demonstrated, and (3) concepts 
presumed ready for transition to development. Each of these 
three classes of SCs serves a different function in the man- 
agement of the Exploratory Development Program. All classes 
of SCs provide a basis for the assessment of a concept's 
military worth; this is an essential prerequisite to the fur- 
ther investment of technological resources (beyond the invest- 
ment required to define concepts sufficiently to permit 


meaningful military worth assessments). 


Bee iedo ss enoysctem concepts. Pre-feasible SCs provide a 


basis for identifying and defining technology which must be 


developed before these concepts become technologically feasible. 
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When communicated to the technical community involved in devel- 
oping broad-based technology, these technological require- 
ments stimulate development of the requisite technology. 
Pre-feasible SCs (and their related requirements for advance- 
ments in technology) are an essential link in the relevance 
audit trail; they make it possible to demonstrate the direct 
relevance of technical effort in the broad-based technology 


category to operational missions. 


Presumed-Feasible System Concepts. SCs for which feasi- 
bility has been presumed but not yet demonstrated provide 
the framework for: (1) defining the requirements for feasi- 
bility demonstrations, and (2) organizing related data. 
Demonstrating the feasibility of an SC can be likened to 
assembling a technological building block kit. As feasibil- 
ity is demonstrated for each essential technology, it is 
added to the kit until all essential building blocks are in 
hand. At that point the concept becomes a "Demonstrated- 


feasible SC" and is ready for transfer to further development. 


Demonstrated-Feasible System Concepts. Demonstrated- 
feasible SCs which are not immediately transferred to devel- 
Opment may be dropped, or, they may be preserved as options 
for future development (depending upon the military worth 
assessment of the system user). Where a decision has been 
made to preserve the option to develop, the SC provides the 
framework for the preservation of that option. The develop- 


ment option is preserved by continually: (1) monitoring 
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relevant technologies, (2) updating the information package 
to include current capabilities, and (3) identifying organi- 


zations which possess the needed capabilities. 


Assessment of the Mali tay Worth of SCs. The CNO and CMC 


are invited to assess periodically the relative military 
worths of all SCs related to each MA objective in their areas 
of warfare. These military worth ratings, termed "V" ratings 
for Value, are to be included in the discussion on MS Objec- 


tives in the Technical Strategy documents. 


Task Area Plans. Detailed plans for accomplishing the 
objectives in approved Program Plans are summarized in Task 
Area Plans (TAP). Since program plan objectives are designed 
to support objectives in approved Technical Strategies, the 
Strategies provide amplifying guidance for the development 


of these more detailed plans. 


PE Oriat ron oyStem., Because pf the direct relationship 
between Technical Objectives called out in the Technical 
Strategies and objectives defined in the Laboratory and SYS- 
COM Program Plans, a relevance audit trail extends from each 
work unit to each assigned Department of the Navy mission. 
When the information system mandated by paragraph 4.b. of 
ASN (R&D) memo of 25 July 1975, "Management of the Exploratory 


Development Program," is developed and in successful operation, 


1t will be possible to make these relevance relationships 
visible through automatic data processing. An ADP query 


Starting with a Mission Need will be able to identify all MA 
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Objectives called out in the MA Technical Strategy for that 
area, all Technology Area Objectives in support of MA Objec- 
tives called out in any of the TA Technical Strategies, all 
program objectives which support the MA Objectives either 
directly or indirectly through TA Objectives, and all work 
units supporting these program objectives. Going the other 
way, ADP capability based on the structure of objectives 
extending upward from the technical program, through the 
Technical Strategies to Department of the Navy missions, will 
make it possible to trace the impact of each work unit on 


higher-level objectives and Mission Needs. 
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APPENDIX C 


STRUCTURE OF THE EXPLORATORY DEVELOPMENT PROGRAM 


The Chief of Naval Development (CND) establishes and 
promulgates the structural framework for planning, programming, 
and budgeting the Navy's Exploratory Development Program. This 
structure provides a link with DOD Program Elements, defines 
the scope of work within specific areas, and identifies the 
organization within the Navy responsible for planning, pro- 


gramming, and budgeting. 


Biome nockoror tie exploratory Development Program. 
The building blocks of the Exploratory Development Program 


are described in the following paragraphs. They are listed 


in descending order, from the largest block to the smallest. 


Program Elements. Program Elements are the smallest sub- 
division of the R&D program considered in the DOD programming 
system. A program is defined as an integrated activity, an 
Meentitfiable military capability; a force, support activity, 
foearch activity, etc., comprising a combination of personnel, 
equipment, and facilities. In exploratory development, a 
Program Element consists of a number of projects in a related 


mora Within a single bucget activity. 


Program Groups. Program groups are primary divisions 
within the Navy exploratory development planning system. They 
are used for management and data-processing purposes, and 


provide the links between the Navy planning system and the 
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Program Elements of the DOD system. They correspond one~to- 


one with the Program Elements--with the exception of Groups 


54 and 55, each of which corresponds to two Program Elements 


and thus requires the project area number to relate a compo- 


nent effort to the correct Program Element. The program 


groups are as follows: 


Program 
Group # Program Element Title 
vl Undersea Target Surveillance 
2 Surface/Aerospace Target Surveillance 
Dy. Command and Control 
ak Missile Propulsion 
a2 Strike Warfare Weaponry 
oe Undersea Warfare Weaponry 
34 Countermeasures 
41 Aircraft 
42 Nuclear Propulsion 
43 Ships, subs and boats 
Sill Biomedical Technology 


eS 


Program 


Element _# 


62 7 12bh 
627 02N 
6272’ 
623554 
O23 52 
62633N 
62734N 
62441N 
62542N 
62543N 


62758N 


Program Program 


Group # Progmamenlement Trete Element # 
52 Ocean and Atmospheric Support 527 5 oN 
Technology 
53 Logistics Technology 62760N 
54 Materials 62761N 
54 Electronic Devices 62762N 
55 Training and Human Engineering 6275 7N 
=e) Personnel Support Technology 62763N 
56 C/B Weapons Defense 62764N 
57 Energy and Environmental Protection 62765N 


one Laboratory IED 62766N 


Functional Area. The functional areas are the five 
principal headings under which exploratory development is 
classified for program planning purposes. These functional 
areas are: 

100 - Target Surveillance 
200 - Command and Control 
300 - Weaponry 

400 - Naval Vehicles 


200 oop Omun Ne cing Logy 


Project Areas. Project areas further subdivide the func- 
tional areas, as shown in Appendix and serve to group 


closely-related task areas. 


Projects. Projects encompass the aggregate of task areas 
under a given project area which fall within a single program 


group (therefore usually within a given Program Element). 
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Subprojects. Subprojects encompass the aggregate of task 
areas within a project which are assigned to a single princi- 
pal development activity. Development activity identifying 


codes are as follows: 


Activity Geode 
NAVAIRSYSCOM W 
NAVELEXS YSCOM X 
NAVFACENGCOM be 
NAVMEDRSCHDEVCOM M 
ONR R 
NAVSEASYSCOM S 
NAVSUPSYSCOM 5b 
DLP Z 


Task Areas. Task areas encompass exploratory development 
effort directed toward a specific objective, e.g., thrust- 
controlled solid rocket. Task areas are the level at which 
detailed plans are prepared. These plans are available to 
the Chief of Naval Development as back-up data for justifica- 
ren Of the Program to higher authorities, if needed. A task 
area may consist of one or more tasks and be assigned to an 
individual Laboratory, Systems Command, or Office of 


implementation. 


Tasks. Tasks are subdivisions of task areas established 
on the basis of organizations performing the work. They are 
used for internal Department of the Navy management only. 


Specific task assignments are made for the portion of the 
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work under a given task area to be contracted or to be 
assigned to an individual Laboratory or field activity. A 
task may consist of one or more work units. 

Work Units. Work units are the smallest segments into 
which research and technology efforts are normally divided 
BemepuUrPOoses Of local administration. In the Exploratory 
Development Program, work units are subdivisions of a task 
selected by the organization performing the work to provide 
effective local technical control and supervision. A work 
Maite 1S technically distinguishable in scope, objective, and/ 
or approach from other scientific or technical efforts with 
which it may be aggregated for financial or administrative 


purposes. 


Numbering System. Based upon the building blocks des- 
cribed above, the numbering system applied to specific devel- 
opment efforts within the Exploratory Development Program is 
designed to permit ready identification and retrieval of infor- 
mation regarding any portion of the Program. An example of 


the use of the system is shown below: 


Pom exploratory Development effort .....F 
Sameme “Missile Propulsion" Program Group .... 3l 
mameeme “Weaponry” Functional Area ........ - 300 
and specific Project Area "Solid Propulsion . .. . 332 


Meme yece Number . . 5. 2. «ww ew tll ltl wl wl lw UWF OBL 332 
Miesportion performed by NAVAIR ....... W 


Paemomoprojgect Number . . . . . » » « « « « « WE 31 332 
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A part of the effort is directed in the 


lawencea Tintisa: Controlled Solid Rocket” .....&. 302 
and has Task Area Number ....... « « « « -.WF3l 332 302 


Within the Task Area are Tasks, and within 

the Tasks are Work Units designated by 

three-character identifiers established 

fsespectively by the SYSCOMs and Labs ......+.. XXX YVY 
So that the entire number identifying a 


Cee weOMDeevOULG DC 2. . 5. . . « » « » WE 31 332 302 xxx yyy 





Work Unit Number 
Task Number 
Task Area Number 
Sub project Number 


Project Number 











W F 31 2 302% if 
1G 7 eee ie 
Devel- Program Program Func— Task Task 
oping (Exp lora- Group tional Area 
Activity tory Devel- (Missile Area (Thrust 
(NAVAIR ) opment ) Propul- (Weaponry) Control- 
sion) led Solid 
Project Rocket 
*The three digit number Area (Solid 
and identifier assigned by Propulsion) 
SYSCOM/Laboratory 


LAL 








Loe 


|e 


ILA 


ILBye 
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